Category: Productivity

Cron expression syntax is built from five space-separated fields — minute, hour, day of month, month, and day of week — that tell a scheduler when to run a job. Each field can hold a value, list, range, step, or *, so 0 9 * * 1-5 runs at 09:00 every weekday.

What Is Cron Expression Syntax?

Cron expression syntax is the compact string format a scheduler uses to decide whether the current clock time matches a schedule. A cron expression describes a set of times, not a countdown. The scheduler compares the current minute, hour, day, month, and weekday against the expression; when every field matches, the command runs.

That difference matters: cron is not an interval timer. Writing */5 in the minute field does not mean “wait five minutes and then run again.” It means “match minute values 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, and 55.” Cron has no memory of the last run and does not measure elapsed time, so the expression is a pattern-matching rule, not a stopwatch. Wikipedia makes the same point plainly: cron is a mere pattern-matcher, not a frequency counter (Wikipedia).

A standard cron line has five time-and-date fields followed by the command to execute. The field order never changes: minute, hour, day of month, month, day of week. System crontabs add a sixth column for the user, and some non-Unix schedulers put seconds first or a year last, but the traditional five-field form is still the baseline. This article focuses on writing the expression correctly. Execution failures, permissions, logging, and notifications are separate operational concerns that come into play only after the schedule itself is right.

Anatomy of 0 9 * * 1-5: every token, decoded

The expression 0 9 * * 1-5 breaks down from left to right. The first field, 0, means minute zero. The second field, 9, means the ninth hour on a 24-hour clock, or 09:00. The third field, *, means every day of the month. The fourth field, *, means every month. The fifth field, 1-5, means Monday through Friday, assuming Sunday is 0 or 7 in the dialect being used. The job fires at 09:00 on weekdays only. In a system crontab, a user column would sit between the fifth field and the command, which is one reason copy-pasting crontab lines between files can shift the meaning.

The Five Fields: Minute, Hour, Day of Month, Month, Day of Week

The five fields are the core reference for cron expression syntax. Each field accepts a limited set of values, and some also accept three-letter names.

Position	Field	Allowed values	Name aliases	Notes
1	Minute	0–59	None	Smallest unit in standard cron
2	Hour	0–23	None	24-hour clock, no AM/PM
3	Day of month	1–31	None	Some dialects add ?, L, W
4	Month	1–12	JAN–DEC	Names may not be supported everywhere
5	Day of week	0–7 or SUN–SAT	SUN–SAT	Sunday numbering varies by dialect

In standard Unix cron, both 0 and 7 mean Sunday, while 1 through 6 mean Monday through Saturday. That convention is not universal. Quartz, AWS EventBridge, Cloudflare, and some other dialects use 1 for Sunday. A copied expression can still parse but run on a different day.

Cron has no concept of a default value. You cannot leave a field blank, and a blank field does not inherit a fallback. The wildcard * is how you say “every value in this field.” Quartz-style schedulers add ? to mean “no specific value,” which helps avoid the day-of-month and day-of-week conflict described later. That difference is often the first sign that a five-field expression is being read by a six-field parser.

Field-order mistakes are costly because they usually produce a valid expression instead of an error. Swap the month and day-of-month fields, and a monthly job can become a daily job on the wrong date. Swap the hour and day-of-month fields, and a 09:00 task can become one that runs on the ninth day of each month. The scheduler does not warn you. It just matches a different pattern.

Value ranges, name aliases, and the values each field rejects

Each field rejects values outside its range, but when that happens varies. Standard cron usually rejects an out-of-range number when the crontab is loaded, while some managed schedulers validate only after deployment. Month names such as JAN and DEC, and weekday names such as MON and SUN, are accepted by many Vixie-derived implementations and are case-insensitive. They are not part of the portable baseline. Vercel’s documentation says Vercel cron jobs do not support alternative expressions like MON, SUN, JAN, or DEC (Vercel, last updated September 16, 2026).

Day-of-week aliases are also where Sunday numbering becomes visible. Cloudflare Workers documentation lists weekdays as 1-7 with case-insensitive three-letter abbreviations such as MON and fri (Cloudflare, last updated September 4, 2026). Quartz and AWS EventBridge also start the week at 1 for Sunday. Unix crontab, Kubernetes CronJob, Jenkins, GitHub Actions, and Spring accept 0 for Sunday and usually also accept 7. A name alias is safer than a bare number when the dialect supports it, but not when the platform rejects aliases entirely.

Field-order mistakes that silently shift your schedule

A field-order mistake rarely causes a syntax error. It causes a different schedule. 0 0 1 * * means midnight on the first day of every month. Reorder it to 0 0 * 1 *, and it means midnight every day in January. The scheduler has no way to know which one you intended.

Two habits prevent most of these errors. First, read the expression aloud with the word “of” between fields, in this order: minute, hour, day of month, month, day of week. Second, check the plain-English translation in a cron expression parser before deploying. A parser that shows the next run times catches a shifted field immediately, because the next-run list no longer matches the cadence you wanted.

The Four Operators: * , – / and How They Combine

Four characters do nearly all the work in standard cron expression syntax: *, ,, -, and /. The asterisk means every allowed value. The comma builds a list. The hyphen defines an inclusive range. The slash defines a step.

Combinations are where the compact syntax becomes powerful. 1-30/5 applies a step of five to the range 1 through 30, producing 1, 6, 11, 16, 21, 26. 0,15,30,45 in the minute field equals */15 in that same field, because both select minutes 0, 15, 30, and 45. 5/15 starts at 5 and steps by 15 through the rest of the field, producing 5, 20, 35, 50. The slash does not create a timer. It selects points from the allowed values.

The highest-frequency standard schedule, * * * * *, runs every minute. According to JasperBernaers.com, that is 1,440 runs per day and 10,080 runs per week. Those numbers are a useful sanity check before deploying anything that runs every minute. A job that sometimes takes longer than sixty seconds can overlap with itself, which is why high-frequency schedules often need a lock or a concurrency policy.

Steps are not intervals: why */7 is not “every 7 minutes”

The step operator is the most common source of false expectations. */7 in the minute field selects minutes 0, 7, 14, 21, 28, 35, 42, 49, and 56. The next run after minute 56 is minute 0 of the following hour, only four minutes later. JasperBernaers.com documents this short-gap behavior: any step that does not evenly divide the field range produces an inconsistent final interval. Minutes divide evenly by 2, 3, 4, 5, 6, 10, 12, 15, 20, and 30. A step of 7 does not divide 60 evenly, so every hour has a short gap. If you need a true seven-minute cadence, standard cron cannot express it; you need a scheduler with second-level precision or an application-level loop.

Which Cron Syntax Are You Actually Writing? A Dialect Matrix

“Cron” is not one language. Before writing an expression, identify the scheduler. The same string can be valid, invalid, or valid with a different meaning depending on the platform.

Platform	Fields	Seconds	Sunday	?	L / W / #	Time zone default
Unix / Vixie crontab	5	No	0 or 7	No	No	Host local or TZ=
Kubernetes CronJob	5	No	0 or 7	No	No	UTC unless spec.timeZone set
Quartz	6 or 7	First	1	Required in one day field	Yes	JVM default unless set
Spring @Scheduled	6	First	0 or 7	Allowed	No	JVM default unless set
AWS EventBridge	6	No; year last	1	Required	Yes	UTC
Azure NCRONTAB	6	First	0	No	No	Service configuration
Cloudflare Workers	5	No	1–7	No	Yes	UTC
Jenkins	5	No	0 or 7	No	No; has H	Controller time zone
GitHub Actions	5	No	0 or 7	No	No	UTC
Vercel	5	No	0–6	No	No	UTC

The practical rule is to treat Linux/Vixie five-field cron as the baseline, then check how the target platform deviates. The biggest deviation is Sunday numbering. In Quartz and AWS EventBridge, Sunday is 1 and Saturday is 7. Copying 0 0 * * 1 from a Linux crontab into Quartz does not fail; it changes the run from Monday to Sunday. On a managed platform, that mistake can run for months before anyone notices.

Is Sunday 0, 1, or 7 in your dialect?

Unix crontab, Kubernetes CronJob, Jenkins, GitHub Actions, and Spring treat Sunday as 0, and most also accept 7. Quartz, AWS EventBridge, and Cloudflare treat Sunday as 1. Azure NCRONTAB uses 0 for Sunday and rejects 7. So the safe answer is not a number at all: if the dialect supports names, use SUN, MON, and so on. If it does not, confirm the numbering from the platform documentation before deploying. Vercel is a notable case where names are not supported, so a numeric day-of-week value must be checked against Vercel’s own 0–6 range (Vercel).

The six-field, seconds-first family: Quartz, Spring, Azure NCRONTAB

Quartz is the clearest example of a different dialect. It is a Java scheduler that uses six or seven fields, with seconds first and an optional year last. Broadcom’s Scheduler documentation, which uses Quartz cron, describes special characters *, ?, #, and #-#, and states that the day of week starts at 1 for Sunday (Broadcom TechDocs, last updated September 17, 2026). Spring @Scheduled uses six fields with seconds first. Azure Functions NCRONTAB also uses six fields with seconds first. Pasting a five-field expression into any of these is not a harmless mistake: the parser either rejects it or shifts every value one position to the right, which can make a daily job run sixty times more often.

What changed recently: OCPS 1.0 (2025) and 2026 vendor documentation

In 2025, the Open Cron Pattern Specification published OCPS 1.0, a formal specification of the Vixie cron dialect that codifies the five-field pattern and rules for previously ambiguous edge cases (Wikipedia). The specification is designed as a backward-compatible superset, with future versions planned to add predefined schedules, second-level precision, and advanced calendar modifiers. Its current role is clarification, not replacement. OCPS is a specification effort, not a runtime that a scheduler automatically follows. Platform behavior still comes from each vendor’s documentation.

That is why vendor documentation dates matter. Vercel’s cron jobs page was last updated September 16, 2026, and Cloudflare’s Cron Triggers page was last updated September 4, 2026. Broadcom’s Quartz-based Scheduler cron syntax page was last updated September 17, 2026. When a platform says Sunday is 1, or that names are unsupported, or that only one day field may be set, the current documentation is the authority for that platform.

Common Cron Schedules You Can Copy — and What They Actually Run

These are standard five-field schedules that work in Vixie-style cron and most managed schedulers that accept five fields. The “runs per cycle” column is the check that catches a misread.

Expression	Meaning	Runs per cycle
*/5 * * * *	Every 5 minutes	12 per hour
0 0 * * *	Every day at midnight	1 per day
0 9 * * 1-5	Weekdays at 09:00	5 per week
0 0 1 * *	First day of every month at midnight	1 per month
0 4 * * 0	Every Sunday at 04:00	1 per week
0 0 1 1 *	Once a year on January 1	1 per year

A real operational schedule shows why spacing matters more than guessing. RE:NODE describes four ordered tasks for a game server: announce a restart at 04:40, save and back up at 04:50, restart at 05:00, and take a weekly locked backup on Sunday at 04:00. The ten-minute gap between the backup start and the restart is intentional. A panel “delay” is a wait time, not a completion check. A backup of a large world can take minutes to finish, so a restart scheduled sixty seconds after the backup begins can interrupt the archive. Two schedules ten minutes apart are easier to reason about than one schedule with a guessed delay.

Copying any of these schedules still requires a dialect check. A line containing 0 for Sunday is safe on Unix and Kubernetes but means something else in Quartz or AWS. A line containing a name alias is readable on Linux and unsupported on Vercel. The expression is portable only after the dialect is known.

Shorthand macros: @daily, @hourly, @weekly, @reboot — and where they don’t work

Some cron implementations support macros in place of the five fields. @yearly or @annually equals 0 0 1 1 *; @monthly equals 0 0 1 * *; @weekly equals 0 0 * * 0; @daily or @midnight equals 0 0 * * *; @hourly equals 0 * * * *. @reboot runs once when the daemon starts, which usually corresponds to a machine boot, and it has no five-field equivalent (Wikipedia).

Macros are convenient in a Linux crontab and unreliable in hosted schedulers. Kubernetes CronJobs work with five-field Vixie syntax and the @daily-style macros, but they do not support @reboot because a cluster has no single boot moment (JasperBernaers.com). GitHub Actions and AWS EventBridge do not support the macros in the same way. If portability matters, write the five fields out. They are longer by a few characters and they work in more places.

The Day-of-Month OR Day-of-Week Rule (The Most Expensive Misreading)

In standard Vixie cron, if both the day-of-month field and the day-of-week field are restricted — that is, neither contains * — the job runs when either field matches. The rule is OR, not AND. This is the most expensive misreading in cron expression syntax because it silently multiplies the number of runs.

The classic example is 0 0 13 * 5. Many people read it as “midnight on Friday the 13th.” In Vixie cron it means “midnight on every 13th, and also midnight on every Friday.” JasperBernaers.com estimates that produces about 60 runs a year rather than one or two. A monthly report written this way can start arriving weekly. A maintenance job written this way can run four or five times a month.

There is no way to express a true AND in standard five-field cron. The workaround is to leave one of the two day fields as * and test the exact condition inside the command. For example, schedule 0 0 13 * * and have the script check whether the day is Friday before proceeding. Some managed schedulers avoid the problem by disallowing both fields at once. Vercel’s documentation states that you cannot configure both day of the month and day of the week at the same time; when one has a value, the other must be * (Vercel). Quartz and AWS EventBridge require a ? in whichever day field is not being used, so only one day condition is ever active.

The self-check is simple: if both date fields are set to something other than *, stop and confirm that OR semantics are actually intended.

Schedules Five Fields Cannot Express: First Monday, Last Day, Friday the 13th

Some schedules look like cron problems but cannot be expressed in five fields. “The first Monday of the month,” “the last day of the month,” and “Friday the 13th” all require a cross-field condition. The first Monday needs a weekday plus a day-of-month range. The last day needs to know the length of the current month. Friday the 13th needs both day-of-month and day-of-week to be true at the same time, which standard cron cannot express because of the OR rule.

Quartz-style extensions add characters for some of these cases. L means last: in the day-of-month field it is the last day of the month, and after a weekday it is the last of that weekday. W means the nearest weekday to a given day, without crossing the month boundary. # followed by a number selects the nth weekday of the month. ? means “no specific value” and is used in one of the two day fields to keep the other active.

Availability is not universal. Quartz, AWS EventBridge, AWS Systems Manager, and Cloudflare support L, W, and # to varying degrees. Standard Linux cron does not. On Linux, the correct approach is to keep the five-field expression as a coarse filter and perform the exact date check inside the command. That also avoids the OR trap, because the expression no longer tries to encode a cross-field rule.

A common mistake on Linux is 0 9 1-7 * 1. It looks like “the first Monday of the month,” but because both day fields are restricted, Vixie cron reads it as “days 1 through 7, OR every Monday.” The job runs far more often than intended. The safer form is a daily or weekly coarse schedule plus a date test in the shell.

L, W, # and ? on Quartz, AWS and Cloudflare

Cloudflare Workers documentation supports five fields with Quartz-like extensions, including L and W in the day-of-month field and L and # in the weekday field (Cloudflare). Its examples include 0 18 * * 6L for 18:00 UTC on the last Friday of the month, and 59 23 LW * * for 23:59 UTC on the last weekday of the month. In Cloudflare’s weekday numbering, 1 is Sunday and 7 is Saturday.

AWS Systems Manager supports L, W, and # in its cron expressions, and its documentation gives cron(15 10 ? * 6L *) as 10:15 AM on the last Friday of every month and cron(0 2 L * ? *) as 02:00 AM on the last day of every month (AWS Systems Manager). The same documentation states that you cannot specify values in both the day-of-month and day-of-week fields; one of them must be ?. Quartz itself requires a ? in one of the two day fields, so only one day condition is active.

The Linux workaround: escaped % and a date test inside the command

On a standard Linux crontab, the workaround for “first Monday” or “last day” is to schedule a superset and filter inside the command. For a first Monday check, schedule the job every Monday at the desired hour and test whether the day of the month is 7 or earlier. For a last-day check, schedule it late in the month and test whether tomorrow is the first day of the next month. The shell expression must be written carefully because % has a special meaning in crontab files.

In a crontab command, an unescaped % turns into a newline, and everything after the first % is sent to the command as standard input. That is why date +%Y-%m-%d inside a crontab can behave incorrectly unless every percent sign is escaped as \%. The cleaner alternative is to put the logic in a script file and call the script from cron. The script runs under a normal shell, where % has no special meaning, and the crontab line stays short. If the command must stay inline, escape each % and test the expression with a parser or a manual run before trusting it.

Does Cron Use UTC or Local Time? Time Zones and DST

Cron schedules wall-clock time in the time zone of the host or service that runs it. The expression itself has no time-zone field. That means two servers with identical cron expressions can run the same job at different real-world moments if their system time zones differ. It also means a hosted scheduler that forces UTC will interpret 0 0 * * * as midnight UTC, which may be 19:00 or 20:00 in the user’s local time.

Several platforms force UTC. Vercel’s cron jobs always use UTC (Vercel). AWS EventBridge cron() rules are always UTC and offer no time-zone option. GitHub Actions scheduled workflows are always UTC. Kubernetes CronJobs default to UTC unless spec.timeZone is set, and that field only exists from Kubernetes 1.27 (JasperBernaers.com). Cloudflare Cron Triggers execute on UTC time (Cloudflare).

Daylight saving time adds two predictable failures. When clocks move forward, an hour disappears. A job scheduled inside that skipped hour does not run at all that day. When clocks move back, an hour repeats. A job scheduled inside that repeated hour runs twice, roughly an hour apart. In the United States, the affected local window is usually 01:00–03:00. A midnight reconciliation job may be safe from the skipped hour, but a job scheduled at 01:30 local can be skipped or duplicated depending on the direction of the clock change.

The skipped hour and the repeated hour

The spring-forward skip is silent. A job at 30 2 * * * simply has no matching wall-clock time on the transition day, so it does not run. No error is raised and no log line is guaranteed. The fall-back repeat is louder in its consequences. A billing run, an email send, or a counter increment that runs twice can produce duplicate charges or duplicate messages. The expression did not change; the clock did.

Mitigations: shift to 03:00, guard with a lock, or pin to UTC

Three practical mitigations reduce DST risk. The first is to move critical jobs outside the 01:00–03:00 local window, with 03:00 or later as a common target. The second is to make the job idempotent and guard it with a lock file so a repeated run does nothing harmful. The third is to schedule in UTC semantics and accept that a “9 AM” job drifts by an hour relative to local office hours twice a year. For machine-facing batch work, UTC is often the more predictable choice. For user-facing notifications, local time may still be worth the complexity, but the schedule should be reviewed against the DST calendar.

crontab -e: Where the Expression Actually Lives

The expression itself is only half of a working cron job. On Linux and Unix-like systems, the expression lives in a crontab file. User crontabs are edited with crontab -e and listed with crontab -l. System crontabs live in /etc/crontab or /etc/cron.d and are managed by system administrators. The behavioral history behind these files is long: cron dates to May 1975, Vixie cron was written in 1987, Vixie cron 4.1 was renamed ISC Cron in January 2004, Red Hat forked it to cronie in 2007, and anacron 2.3 was merged into cronie in 2009 (Wikipedia). Those names matter because “cron” behavior on a modern Linux distribution often comes from a Vixie-derived implementation plus later patches.

Cron’s environment is not the same as an interactive shell. The PATH is usually minimal, often /usr/bin:/bin, and shell profile files are not read. A command that works when typed into a terminal may fail with “command not found” under cron. Use absolute paths for binaries and scripts, or set a PATH= line at the top of the crontab. Output is another common silent failure point. Cron may try to mail stdout and stderr to the crontab owner, and if no mail system is configured, that output disappears. Redirect output explicitly with >> /var/log/job.log 2>&1.

To test the environment cron will use, run the command through env -i /bin/sh -c '...'. That reproduces the stripped environment and catches missing variables, relative paths, and profile-dependent assumptions. Plain cron also does not catch up on runs missed while the machine was off. If the machine was asleep or powered down when the scheduled minute passed, that run never happens. Anacron and systemd timers with Persistent=true exist to cover that gap, but they are not the same as standard cron.

System crontab’s extra user column

System crontabs add a user field before the command. A line in /etc/crontab or /etc/cron.d has six columns: five time-and-date fields, the user to run as, and the command. User crontabs do not have that column. Copying a five-field user crontab line into a system crontab shifts the command into the user field and can make the line fail or run as the wrong user. The reverse mistake is just as common: copying a six-column system line into crontab -e makes cron treat the username as the start of the command.

Before You Deploy: A Sanity-Check Checklist

A cron expression that parses is not necessarily a cron expression that does what you meant. Deployment checks should verify the next run time, the time zone, the overlap behavior, and the missed-run policy.

First, confirm the dialect and the forced time zone. A five-field expression in a six-field scheduler is either rejected or shifted. A UTC platform changes what “9 AM” means. Second, parse the expression and inspect the next several run times rather than only checking that the parser accepts it. Tools such as crontab guru and an online cron parser can show the next fire times, which is where field-order errors and */7 short gaps become visible. Third, check both day fields. If day-of-month and day-of-week are both restricted, confirm that OR behavior is intended. Fourth, decide whether the job can overlap and protect it if it can.

Verify per platform: runnable commands

Different platforms have different verification paths.

• Linux crontab: test the command environment with env -i /bin/sh -c '/path/to/job.sh'. This catches missing PATH entries and profile dependencies.
• systemd timer: run systemd-analyze calendar 'OnCalendar=...' to print the next elapse time before enabling the timer.
• Kubernetes CronJob: run kubectl create job --from=cronjob/<name> manual-run-1 to start the job immediately without waiting for the schedule.
• Cloudflare Workers: use wrangler dev and call the local scheduled route at /cdn-cgi/local/scheduled to test the handler. The documentation shows curl "http://localhost:8787/cdn-cgi/local/scheduled" and supports a ?cron= parameter to simulate a pattern (Cloudflare).
• Parser check: crontab guru or cron-parser can translate the expression and list upcoming runs. Verify the next run time, not just whether the expression parses.

Stop overlapping runs and spread load: flock, Jenkins H, RandomizedDelaySec

If a job can take longer than its interval, prevent overlapping runs. On Linux, wrap the command in flock -n /tmp/job.lock, which exits immediately if the lock is already held. On Kubernetes, set concurrencyPolicy: Forbid on the CronJob so a new run does not start while the previous one is still active. In application code, an advisory lock or a lock file can do the same job.

High-frequency schedules also create load spikes when many jobs fire at the same minute. Jenkins uses H to substitute a hashed value, so each job runs at an unspecified but invariant time within the hour instead of all jobs starting at minute 0 (Wikipedia). systemd timers use RandomizedDelaySec to jitter starts across a fleet. Both approaches spread load without changing the underlying cadence. For missed runs, remember that cron does not catch up. Anacron and systemd timers with Persistent=true do, but their catch-up timing differs, so the behavior should be documented alongside the schedule.

Two final checklist items belong at the top: confirm whether the platform is local time or UTC, and confirm whether both date fields are restricted. Those two checks prevent a large share of “the job ran at the wrong time” and “the job ran too often” incidents.

Kubernetes CronJobs and GitHub Actions: Same Syntax, Different Rules?

Kubernetes CronJobs and GitHub Actions scheduled workflows both accept five-field Vixie-style cron expressions, but the syntax is where the similarity ends. The differences are operational guarantees, not field definitions.

Kubernetes CronJob syntax matches crontab for the five fields and supports the @daily-style macros, but it does not support @reboot because a cluster has no single boot moment. The default time zone is UTC unless spec.timeZone is set. concurrencyPolicy controls whether a new run can start while the previous one is still active; Forbid prevents overlap. startingDeadlineSeconds controls how late a missed run may still start. If a CronJob misses too many schedules, the controller can give up on it, and the reason is recorded as an event. Those are runtime behaviors layered on top of the expression.

GitHub Actions scheduled workflows are best effort. They can be delayed by 5 to 30 minutes, especially at busy times such as the top of the hour. They run in UTC only. Intervals shorter than five minutes are not honored. A scheduled workflow is also disabled automatically after 60 days without repository activity (JasperBernaers.com). A correct expression therefore does not guarantee an on-time run. For jobs that must happen on time, trigger them from a system that guarantees delivery and use Actions as the runner rather than the scheduler.

Kubernetes CronJob: UTC default, concurrencyPolicy, startingDeadlineSeconds

The Kubernetes CronJob spec accepts the standard five fields. The default schedule interpretation is UTC, and spec.timeZone is only available from Kubernetes 1.27. concurrencyPolicy: Forbid is the built-in way to stop overlapping runs, which is often preferable to a lock file inside the container. startingDeadlineSeconds defines the window after the scheduled time in which a missed run may still start. If the controller misses too many schedules in a row, it stops creating jobs and records an event. The expression can be perfectly correct while the runtime still skips or delays a run.

GitHub Actions: best-effort timing, UTC only, disabled after 60 days

GitHub Actions scheduled workflows use five-field cron syntax, but they are explicitly best effort. Delays of 5 to 30 minutes are normal, and runs can be dropped at peak times. Schedules shorter than five minutes are not honored. The time zone is always UTC. Scheduled workflows are disabled after 60 days without repository activity, so a dormant repository can silently stop running its schedule. These are platform policies, not syntax errors.

Managed Schedulers Beyond Linux: Vercel, AWS, and systemd Timers

When moving from a Linux crontab to a managed scheduler, check three things before copying the expression: whether name aliases are supported, whether both day fields can be set, and whether the time zone is forced. Those three checks catch most migration failures.

Vercel uses five fields but does not support MON, SUN, JAN, or DEC. It does not allow both day-of-month and day-of-week to be configured; one must be *. It always uses UTC. Each invocation includes an x-vercel-cron-schedule header containing the cron expression that triggered it, which helps distinguish schedules when multiple cron jobs share the same path (Vercel).

AWS EventBridge uses six fields with a year field last, requires ? in one of the two day fields, treats Sunday as 1, and always uses UTC. AWS Systems Manager has two different cron modes. Maintenance windows require six fields and support the fastest rate of five minutes. Associations require five fields and must use an interval of at least 30 minutes; AWS states that associations cannot use cron or rate expressions shorter than that. Maintenance windows support L, W, and #, while associations also support # and L in their documentation examples (AWS Systems Manager).

systemd timers are the server-side alternative to cron. They use OnCalendar, support Persistent=true for catch-up, and support RandomizedDelaySec for jitter. systemd-analyze calendar verifies the schedule before the timer is enabled. The trade-off is real: a managed platform may give stronger delivery guarantees but removes some syntax freedom, while a server crontab keeps more syntax freedom but leaves overlap protection and catch-up behavior to the operator.

Vercel Cron Jobs: no name aliases, one day field, always UTC

Vercel supports five fields with minute, hour, day of month, month, and day of week. It does not support name aliases. It does not allow both day-of-month and day-of-week to be restricted. It always uses UTC. The invocation header x-vercel-cron-schedule tells the function which expression triggered it. Those constraints make Vercel simpler in one way — the OR trap cannot occur — and stricter in another: an expression copied from a Linux crontab that uses MON or JAN will not work.

AWS EventBridge and Systems Manager: six fields, ?, Sunday = 1

AWS EventBridge cron() expressions use six fields with a year at the end, require ? in exactly one of the day fields, and use Sunday as 1. They are always UTC. AWS Systems Manager maintenance windows require six fields and support a minimum rate of five minutes. Systems Manager associations require five fields and support a minimum interval of 30 minutes, and they do not currently support specifying months. Both AWS paths support advanced day syntax to different degrees, so the correct reference is the specific AWS service documentation, not a general cron guide.

systemd timers: OnCalendar, Persistent, RandomizedDelaySec

systemd timers use a different calendar syntax, not cron syntax. OnCalendar defines the schedule, Persistent=true runs a missed job after the machine comes back, and RandomizedDelaySec adds jitter so a fleet does not start every job at the same second. systemd-analyze calendar prints the next elapse time and is the verification step before enabling the timer. One semantic difference matters when migrating: cron treats day-of-month and day-of-week as an OR when both are restricted, while systemd treats them as an AND. A schedule that uses both fields can fire on fewer days as a timer than it did as a crontab line.

Conclusion

Cron expression syntax comes down to five fields and four operators, but the errors that cost real time usually come from outside the grammar. Sunday numbering changes between dialects, the day-of-month and day-of-week fields combine with OR semantics in Vixie cron, and time zones and daylight saving can skip or repeat a run without any syntax mistake. Before writing the first expression, identify the scheduler and the forced time zone. Before deploying, use a cron parser or a platform-specific command to verify the next run time, then protect the job with a lock or concurrency policy if it can overlap. The expression is only one part of a reliable schedule.

FAQ

Is Sunday 0 or 7 in a cron expression?

Unix, Kubernetes, Jenkins, GitHub Actions, and Spring accept 0 for Sunday, and most also accept 7. Quartz, AWS EventBridge, and Cloudflare use 1 for Sunday, where 0 is either invalid or means something different. The safe approach is to confirm the dialect before using a number, or use a name alias such as SUN when the platform supports it. Vercel does not support name aliases, so its numeric range must be checked directly.

Does cron use UTC or the local time zone?

The expression itself carries no time zone. Cron matches the wall-clock time of the host or service running it. Vercel, AWS EventBridge, GitHub Actions, and Kubernetes CronJobs default to UTC. A Linux crontab uses the server’s local time unless a TZ= line overrides it. If the job matters, confirm the time zone before converting local intentions into an expression, or deliberately schedule in UTC and accept the seasonal drift.

Why does my cron job run more often than I expected?

The most common cause is the day-of-month and day-of-week OR rule. When both fields are restricted, Vixie cron runs the job when either matches, so 0 0 13 * 5 fires on every 13th and every Friday. The second common cause is a step that does not divide its range, such as */7 in the minute field, which creates a short gap at the end of each hour. Also check for duplicate crontab entries or repeated deployments.

Can cron run a job every 30 seconds?

Standard five-field cron cannot. Its smallest unit is one minute, and there is no seconds field. Schedulers that support seconds use a six-field format with seconds first, including Quartz, Spring, and Azure NCRONTAB. On a plain Linux server, the usual workaround is a one-minute job that loops internally with sleep, or a systemd timer with OnUnitActiveSec. GitHub Actions does not honor intervals shorter than five minutes.

What does */5 * * * * mean, and what does 5 mean instead?

*/5 in the minute field selects minutes 0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, and 55 — twelve runs per hour. A plain 5 selects only minute 5 of each hour — one run per hour. The difference is that */n steps through the allowed values rather than measuring an interval, so a step that does not divide 60 evenly produces an inconsistent final gap.

How do I schedule “the first Monday of the month” or “the last day of the month”?

Five-field cron cannot express those cross-field conditions directly. Quartz, AWS, and Cloudflare support # for the nth weekday and L for the last day. On Linux, schedule a coarse run and test the date inside the command, escaping % as \% if the crontab line uses date formatting. Avoid 0 9 1-7 * 1 on Vixie cron, because the OR rule makes it run on days 1–7 and every Monday, not only the first Monday.

A token parser turns an input string into discrete tokens and interprets them: lexical tokens in source code, Base64url segments in a JWT, placeholders like {SEQ:X} in a document template, or design tokens in a build pipeline. Which parser you need depends on the format you’re decoding — and the term still spans four unrelated meanings that no single page explains.

What Is a Token Parser?

A token parser is the component that takes a raw string and produces a sequence of typed, interpreted tokens. It answers three questions, in order: where does one meaningful unit end and the next begin, what kind of unit is it, and what does it mean once you know its type? Everything downstream — an authorization header, an abstract syntax tree, a generated document number, a compiled design system — inherits whatever the parser decided.

That definition also marks the parser’s boundary. A token parser is not a validator, a verifier, or a serializer. Parsing ends where trust decisions begin. Decoding a JWT’s payload and confirming the token was signed by a party holding the private key are different operations, run by different code, with different failure modes. Conflating the two is the most common defect in this space.

The four variants in this guide share the same skeleton, even though they operate on completely unrelated formats:

• A scanner or tokenizer that walks the input and segments it
• A token grammar — a regex, a DFA, or a lookup registry — that decides which segment matches which token type
• A token record carrying at minimum a type, the raw lexeme, an interpreted value, and a position for error reporting
• An error path that defines what happens when the input does not match the grammar

That ambiguity is the real problem. Search for “token” today and you’ll find at least five meanings: a lexeme in source code, a Base64url segment inside a JWT, an OAuth access token presented in an Authorization header, a design token in a build pipeline, a template placeholder — and, in developer tooling, a counted unit of LLM consumption. The premise of the ezparser JWT parser, for example, is that “token” means a signed JSON credential. The premise of the plus-uno token-literal check is that “token” means a design-system value. Both are called token parsers. They don’t read the same thing.

This matters because picking the wrong branch gives you a parser that compiles, runs, passes a smoke test, and reads the wrong object. Point a JWT-shaped parser at a design-token file and it won’t throw — it will simply extract nothing useful, or extract something plausible but wrong.

Token parser vs. tokenizer vs. lexer: where the lines are

These terms overlap, but they describe different stages. A tokenizer or lexer produces the token stream from a character string. A token parser consumes that stream and assigns structural meaning — grouping tokens into expressions, blocks, or records.

In practice, plenty of codebases use the words interchangeably, and the distinction often collapses inside a single module. The useful rule: check what the library actually accepts and returns before assuming which term it means. If it takes a string and returns a list of typed units, the vocabulary is cosmetic. If it takes a string and returns a tree, it’s doing both jobs, and the naming isn’t worth arguing about.

Quick glossary: token, lexeme, claim, Base64url, design token, placeholder

• Token — a typed unit produced by parsing; its type, not its text, drives downstream behavior.
• Lexeme — the raw substring a token was matched from, kept for error messages and round-tripping.
• Claim — a name/value pair inside a JWT payload, such as iss, sub, or exp.
• Base64url — a URL-safe Base64 variant that substitutes - for + and _ for /, and typically omits padding.
• Design token — a named design value (color, dimension, typography, motion) held in a platform artifact and consumed by a build pipeline.
• Placeholder — a bracketed token inside a template string, such as {BRANCH} or {SEQ:X}, replaced at generation time.

Which Token Parser Do You Actually Need? A Decision Table

The input in front of you determines which parser you should reach for. Find your input in the first column, then read across — the rest of this guide follows this table.

Input you have	Token meaning	Output you want	Parser type	First thing to get right
A three-part dotted string copied out of a browser or log	JWT / OAuth security token	Readable header and claims	Base64url segment decoder + JSON deserializer	Decode only — never verify, never trust for authorization
Source code, a log line, or a configuration file	Lexical token	A token stream feeding a parser or AST	Tokenizer / lexer	Maximal munch and rule ordering
A string containing {PREFIX} or {SEQ:X}	Template placeholder token	A substituted, unique output string	Registry-driven substitution	Fail loud on unknown tokens
JSON, SCSS, or Kotlin artifacts holding colors, spacing, typography	Design token	Generated platform code	Build-pipeline generator with a pinned source	Pin the source commit and reproduce byte for byte

Every row works on a different contract. The JWT row is inspection only — the output goes to a human debugging a login flow. The lexical row is a pipeline stage. The template row is a substitution engine where correctness means uniqueness, not readability. The design-token row is a code generator where correctness means diff size.

None of the pages that currently rank for this term defines it or separates those four meanings. They’re either tool landing pages describing one branch in isolation, or GitHub issues that assume you already know which branch you’re on. The table above fills that gap.

One-line rule: the format you are decoding picks the parser

If you remember nothing else, remember this: the format you’re decoding picks the parser, not the other way around. You don’t choose a token parser and then go looking for something to feed it. Name the token format first, and the format dictates the grammar, the failure behavior, and where the parser has to stop.

Token Parser for Security Tokens: How to Decode and Inspect a JWT

The JWT parser is the most-searched branch of this topic, and the one where misuse causes the most damage. The procedure below is for inspection only. It’s meant for a debugger, a local script, or a CLI — never for a server-side authorization decision.

Step 1 — split on the dots. RFC 7519 defines a compact JWT as three Base64url segments joined by periods: Header, Payload, and Signature. Splitting gives you three strings. Anything other than three is a malformed token.

Step 2 — Base64url-decode the first two segments. Base64url substitutes - for + and _ for /, and the padding is usually stripped. A standard Base64 decoder will fail on a stripped segment unless you re-pad it to a multiple of four characters. This is the most common reason a naive decoder throws on a valid token.

Step 3 — read the fields that matter. In the header, alg tells you which signing algorithm was declared, and typ usually reads JWT. In the payload, iss is the issuer, sub is the subject, aud is the intended audience, iat is issued-at, and exp is expiry — all timestamps in Unix seconds. Convert exp to a UTC time and compare it against the current clock; that answers “is this token stale?” without a round-trip to the issuer.

Step 4 — treat the signature segment as opaque bytes. Without the HMAC secret or the RSA public key, the third segment tells you nothing. It’s a byte string whose only purpose is to be checked by code that holds the key.

Step 5 — write the decoder. The following examples are explicitly decode-only.

import base64, json, datetime

def decode_segment(segment: str) -> dict:
    # Base64url: restore padding before decoding
    padded = segment + "=" * (-len(segment) % 4)
    raw = base64.urlsafe_b64decode(padded.encode("ascii"))
    return json.loads(raw.decode("utf-8"))

def inspect(jwt: str) -> None:
    parts = jwt.split(".")
    if len(parts) != 3:
        raise ValueError("compact JWT must have exactly three segments")
    header, payload, _signature = parts  # signature is opaque here
    header = decode_segment(header)
    payload = decode_segment(payload)

    print("alg:", header.get("alg"), "typ:", header.get("typ"))
    for claim in ("iss", "sub", "aud", "iat", "exp"):
        print(f"{claim}: {payload.get(claim)}")

    if "exp" in payload:
        expiry = datetime.datetime.fromtimestamp(payload["exp"], datetime.timezone.utc)
        print("expires at (UTC):", expiry.isoformat())
        print("expired:", expiry < datetime.datetime.now(datetime.timezone.utc))

The Node equivalent is shorter because the runtime already understands the encoding:

// decode-only — this does not verify the signature
function inspect(jwt) {
  const parts = jwt.split('.');
  if (parts.length !== 3) throw new Error('compact JWT must have exactly three segments');

  const [header, payload] = parts.map((segment) =>
    JSON.parse(Buffer.from(segment, 'base64url').toString('utf8'))
  );

  console.log('alg:', header.alg, 'typ:', header.typ);
  for (const claim of ['iss', 'sub', 'aud', 'iat', 'exp']) {
    console.log(claim, payload[claim]);
  }
  if (payload.exp) {
    const expiry = new Date(payload.exp * 1000);
    console.log('expires at (UTC):', expiry.toISOString());
    console.log('expired:', expiry < new Date());
  }
}

Step 6 — place the token in its OAuth context. In OAuth 2.0, an access token arrives in an Authorization: Bearer <access_token> header. The token_type value issued alongside the token is what tells a client how to present it — not an assumption baked into the calling code.

Splitting and Base64url-decoding the three segments

The padding problem deserves its own note, because it produces confusing failures. Base64url encoding of a JSON object rarely lands on a multiple of four characters, and most producers strip the = padding rather than emit it. A decoder that expects padded input either throws or silently truncates. Re-padding with (-len(segment) % 4) characters — as in the Python example above — restores the input to a form the standard decoder will accept.

The second trap is the alphabet. - and _ aren’t valid characters in standard Base64. A decoder that doesn’t switch alphabets will reject a perfectly valid segment. Python’s base64.urlsafe_b64decode and Node’s 'base64url' encoding handle this automatically; a hand-rolled decoder usually won’t.

Reading exp, iss, aud and sub without a server round-trip

Four claims carry most of the debugging value. exp tells you whether the token is still current. iss tells you which system minted it — the usual cause of a rejected token is that it came from a staging issuer when production was expected. aud tells you which API it was meant for, and a token minted for one audience will be correctly rejected by another. sub tells you who the token is about.

All four can be read offline, with no key required. That’s the point: claims are informative, not authoritative. Anything a decoder can read, an attacker can write.

Where a Token Parser Must Stop: Decoding Is Not Verification

Decoding reverses Base64url and deserializes JSON. That’s the whole operation. Anyone can craft a token with any claims they like, sign nothing, and produce a string that decodes perfectly. Only signature verification with the HMAC secret or RSA public key establishes that the claims actually came from the issuer. A token parser that returns claims to a caller who then treats them as trusted isn’t misconfigured — it’s been asked to do something it can’t do.

The boundary generalizes across credential formats. A SAML response decoder shows every field of an assertion without proving it came from the identity provider, and a cookie header parser shows what the browser sent without vouching for the session behind it. In each case, parsing makes claims readable; only verification makes them trustworthy.

OAuth 2.0 makes that boundary explicit. RFC 6749 §5.1 requires both access_token and token_type in a successful token response, and states that token_type is case-insensitive — so Bearer, bearer, and BEARER are all valid. RFC 6749 §7.1 is the rule that gets skipped in practice: a client must not use an access token whose type it doesn’t understand.

Reject a missing token_type instead of defaulting to Bearer

The cortexfs issue #282 documents exactly this failure. In parse_oauth_token_response, validation used Option::is_some_and, which only rejects a present non-Bearer value. A response with no token_type field at all passed validation silently. Downstream, model_headers serialized every Credential::OAuth as Authorization: Bearer <access_token> — so an undeclared type got reinterpreted as Bearer by default.

The shape of the fix generalizes well beyond one repository. Require the field to be present. Accept Bearer case-insensitively. Keep rejecting unsupported types such as mac. Update the success fixtures in the same change to include token_type: "Bearer". As the issue records, the expected production change was one existing predicate tightened in place, netting zero production lines of code — the fix removed an implicit assumption instead of adding an abstraction.

Here’s the pattern to internalize: a validator that checks “if present, must be valid” is a weaker thing than one that checks “must be present and valid.” The first admits a missing field; the second doesn’t. When the consuming code has a fallback default, the first becomes a silent policy decision made by omission.

Keep parser errors bounded and generic

A token parser must never return its internals to a client. Offsets, grammar rule names, expected token types, partial parse states — that’s a map of the input format, and handing that map to an untrusted caller is a gift.

The synchro issue #53 records this as audit finding C12, independently verified with a verdict of CONFIRMED at severity Low. The server returned token parser details in responses, and no existing test covered the behavior. The proposed fix is to return a bounded generic error instead of parser internals, with an acceptance criterion that a permanent test fails without the fix and passes with it. Detailed diagnostics belong in server-side logs behind a stable correlation ID — the client gets an opaque error and a handle, not a description of the grammar.

Building a Lexical Token Parser (Tokenizer or Lexer)

For source code and structured text, the pipeline runs: raw string → scanner → token stream → parser → AST. The token parser is the stage that decides where one unit ends and the next begins, and every stage after it inherits those decisions.

Define the token record first. A workable minimum has four fields: an enum type, the raw lexeme, an optional interpreted value (a parsed number, an unescaped string), and a line/column position for diagnostics. Skipping the position field is the most common shortcut and the one that hurts most later — error messages without positions are close to useless.

Three implementation rules prevent most tokenizer bugs:

1. Maximal munch. Always take the longest valid match at the current position, so == is one token rather than two =.
2. Order rules so keywords precede identifiers. If identifiers are matched first, while becomes an identifier and the keyword rule never fires.
3. Decide explicitly about whitespace and comments. Either emit them as token kinds or skip them deliberately — but make it a documented decision, not an accident of the first regex that happens to match.

A working hand-written lexer fits in roughly forty lines and covers the token kinds most languages need:

import re
from dataclasses import dataclass

@dataclass
class Token:
    type: str
    lexeme: str
    value: object = None
    line: int = 1
    col: int = 1

# Order matters: keywords before identifiers, longest operators first.
RULES = [
    ("NUMBER", r"\d+(\.\d+)?"),
    ("IDENT",  r"[A-Za-z_]\w*"),
    ("STRING", r'"(?:[^"\\]|\\.)*"'),
    ("OP",     r"==|!=|<=|>=|&&|\|\||[-+*/=<>(){},;]"),
]
KEYWORDS = {"if", "else", "while", "return"}
SKIP = re.compile(r"[ \t\r]+")

def tokenize(src: str) -> list[Token]:
    tokens, i, line, line_start = [], 0, 1, 0
    while i < len(src):
        if src[i] == "\n":
            line, i, line_start = line + 1, i + 1, i
            continue
        m = SKIP.match(src, i)
        if m:
            i = m.end()
            continue
        for kind, pattern in RULES:
            m = re.compile(pattern).match(src, i)
            if not m:
                continue
            lexeme = m.group(0)
            if kind == "IDENT" and lexeme in KEYWORDS:
                kind = "KEYWORD"
            value = None
            if kind == "NUMBER":
                value = float(lexeme) if "." in lexeme else int(lexeme)
            elif kind == "STRING":
                value = lexeme[1:-1]
            tokens.append(Token(kind, lexeme, value, line, i - line_start + 1))
            i = m.end()
            break
        else:
            raise SyntaxError(f"unexpected character {src[i]!r} "
                              f"at line {line}, column {i - line_start + 1}")
    tokens.append(Token("EOF", "", None, line, i - line_start + 1))
    return tokens

if __name__ == "__main__":
    for t in tokenize('total = price * 2; if total > 100 { return "big"; }'):
        print(f"{t.type:<8} {t.lexeme!r:<12} {t.value!r:<8} {t.line}:{t.col}")

Printing the token dump isn’t decoration. The dump is what you compare against when a grammar change produces unexpected behavior, and it separates “the lexer is wrong” from “the parser is wrong” in seconds instead of hours.

Maximal munch, rule order and keyword ambiguity

Maximal munch and rule order are the same problem looked at from two angles. A rule set that matches = before == produces two EQ tokens where one EQEQ was intended, and the parser downstream reports a confusing error two stages away from the cause. Sorting operator alternatives longest-first, and keywords before identifiers, eliminates both classes of bug without any lookahead machinery.

Keyword ambiguity needs an explicit decision. Most languages treat keywords as reserved: the lexer recognizes if as a keyword in every position, so a variable named if is a syntax error rather than an identifier. Languages that allow contextual keywords push the decision into the parser instead. Either choice works; leaving it undecided gives you a lexer whose behavior depends on which rule happened to be listed first.

Does tokenization change parser accuracy? Evidence from a 2026 study

Tokenization quality isn’t cosmetic, and there’s now published evidence for that. Shamaeva and Loukachevitch (2026), in Pattern Recognition and Image Analysis, Vol. 36, pp. 486–497 (DOI 10.1134/S105466182670029X), compared two ways of evaluating a syntactic parser: with its built-in tokenizer, and with a tokenizer that returns gold markup. For a significant number of sentences, the built-in tokenization differed from the gold one, and average UAS and LAS scores were higher when the parser was evaluated with gold markup — some metrics by 0.05 or more.

The study covers Russian-language corpora (SynTagRus, GSD, PUD, Taiga, and Poetry) and the parsers UDPipe, Stanza, Natasha, DeepPavlov, and spaCy, as they existed for this 2026 research. Those are the versions studied — not a claim about current releases of any of those tools.

The practical takeaway for anyone building a parser: if your accuracy numbers look wrong, test the tokenizer before you rewrite the grammar. A meaningful share of apparent parsing errors originate one stage earlier, in segmentation decisions the parser never sees.

Parsing Template Tokens and Placeholders Safely

A document-numbering engine is a good concrete case. The tan-erp issue #6, opened 18 September 2026, asks how IDocumentNumberGenerator should parse template strings containing nine token forms — {PREFIX}, {BRANCH}, {YYYY}, {YY}, {BBBB}, {BB}, {MM}, {DD}, and {SEQ:X} — derive a period key from them, and generate unique document numbers atomically with full unit-test coverage.

Four design decisions follow from that framing.

Design around a registry, not a loose regex. A table of known token names with a resolution function per token makes unknown placeholders detectable. A single permissive pattern like \{[A-Z]+\} matches anything and tells you nothing about whether the result is meaningful.

Anchor the pattern. Match whole placeholders with anchored boundaries, and handle escaping explicitly. If a substituted value can itself contain {, an unanchored second pass will re-parse it — that’s how injection-shaped bugs get into template engines.

Fail loudly on unknown tokens. Raising is the correct behavior. Leaving the literal {BRANCH} in the output, or dropping it, silently produces malformed or colliding identifiers, and the failure surfaces much later in a way that’s hard to trace back to the template.

Keep atomicity outside the parser. Parse first, then generate the counter inside a single transaction. The parser can be perfectly correct and two concurrent requests can still get the same number if the sequence increment isn’t atomic.

There’s a harder adjacent case worth flagging: languages without whitespace word boundaries. The same issue pairs the template work with Thai tokenization, because a naive split on spaces produces no segmentation at all for Thai text. That branch needs a dictionary- or rule-based tokenizer — a different algorithm from placeholder substitution, and the two shouldn’t be combined into one pass.

Unknown placeholders should raise, not fall through

The difference between “raise” and “fall through” is the difference between a caught error and a data-integrity incident. A fall-through leaves the literal token in a generated document number, and every request that hits the same template produces the same literal — so the collision is systematic, not random. Raising converts invisible corruption into a stack trace at the first occurrence.

Token Parsing in a Design-Token Build Pipeline

A design-token parser reads generated platform artifacts — SCSS declarations, var() references, Kotlin token files — rather than a hand-maintained source file. That changes the risk profile: the input is machine-written, verbose, and changes shape whenever the upstream generator changes.

The slint issue #3, opened 27 September 2026, documents the parsing contract for a Material 3 Expressive token pipeline. The source is Compose material3’s generated token files in androidx/androidx, pinned to commit 23327507f7fc7d5b19d65fec4b090f60c970079b (androidx-main, 2026-09-27), whose files carry the header // VERSION: 14_1_0. Since 2026-08-05 those files use inline value classes, so token values sit on get() = lines — a parser written for the older file shape silently reads nothing and reports no error.

Three rules from that contract transfer to any design-token pipeline:

1. Fail loudly on any token that cannot be parsed or mapped. Never silently drop or default a value. A dropped token becomes a component that renders with the wrong color, discovered by a designer rather than by CI.
2. Reproduce committed output byte for byte in CI from the pinned commit. The generator runs on a clean checkout with one documented command, and the check passes only when the output is identical.
3. Treat a version-pin bump as a deliberate, reviewed change. The pin is recorded in one place and printed into generated file headers, so bumping it is an explicit diff rather than an incidental drift.

The referenced pipeline also resolves references between token files — a button token pointing at ShapeKeyTokens.CornerMedium, which points at ShapeTokens.CornerMedium — so the parser needs a typed intermediate model, not a flat name/value map.

Why the pinned commit and the date are the parsing contract

A design-token parser is only correct relative to a specific input. The same file path yields different values at different commits, and the file format itself changed on 2026-08-05. Recording 23327507f7fc7d5b19d65fec4b090f60c970079b and // VERSION: 14_1_0 in one place turns “the parser is broken” into “the pin moved and the diff is these lines.” Without the pin, a format change and a parser bug look identical from the outside.

Build the Token Sets Once: Parsing Performance and Duplication Risk

Token sets, DFAs, and grammar patterns should be constructed once and held on the parser instance. Rebuilding them at every call site is a performance problem, and duplicated parsers are a correctness problem that costs more than the performance one.

The measured cost of getting the performance case wrong comes from the hefermotor issue #22, opened 19 September 2026. The _TokenSets primitive built an array literal on every method call, and the grammar called those methods at roughly 35 test sites. For the three largest sets — expr_start with 40 tokens, called at every call expression, plus case_pattern_start and type_start — building per call added 10–15% to the sum of Parse.tree over the standard library’s files in a debug build. Moving those three to fields on _Parser removed the overhead. The remaining, smaller sets were not measured.

The correctness case comes from the plus-uno issue #622, from the 2026-09-18 architecture review. The docs token-literal check carried a parallel implementation of most of the tokens module: its own token reader, declaration regex, alias resolver with its own cycle guard, colour key, dimension key, family map, and SCSS brace parser. Removing the duplicate — tracked in PR #667 — affected 315 live token pairs that were equal before the change and unequal after. Unchanged output had been resting on no fallback literal pairing an opaque hex against a translucent token, not on any guarantee the code provided.

That number is the argument. A duplicate parser isn’t just redundant; it’s a second set of semantics that diverges the moment either copy is edited, and the divergence stays invisible until someone enumerates it.

Enumerate output changes instead of assuming they are absent

The plus-uno acceptance criteria state the discipline directly: any output change is enumerated and reviewed, not assumed absent. That’s a higher bar than “tests pass,” because passing tests only prove the cases they cover. The 315-pair review is the model — the reviewer had to name the mechanism that preserved the output, not just observe that nothing appeared to break.

The rule of thumb: one grammar module, one token registry, built once, version-pinned.

How to Test a Token Parser: Acceptance Criteria from GitHub Tickets

This section exists because of what the search results show about how the work is actually gated. Eight of the top ten results are GitHub issues and repositories, and five of them gate completion on tests passing or on enumerated acceptance criteria — but none presents those criteria as reusable guidance. The six criteria below are pulled from those tickets and generalized.

1. A permanent regression test that fails without the fix and passes with it. The synchro issue #53 attaches this explicitly to the error-leak fix for audit finding C12, noting that no existing test covered the behavior. A test written after the fix that passes on the fixed code proves nothing on its own; it must be shown to fail on the unfixed code.

2. Enumerate every output difference. The plus-uno criteria require that output changes be enumerated and reviewed rather than assumed absent. Claiming “no behavior change” without naming the mechanism is not evidence.

3. Byte-for-byte reproduction in CI from a pinned upstream commit. The design-token pipeline’s reproducibility check is the strongest of the six because it admits no interpretation: either the generated bytes match or they do not.

4. Fixture hygiene in the same change. When you tighten a predicate, update the success fixtures alongside it. The cortexfs fix added token_type: "Bearer" to the Codex OAuth success fixtures in the same change that required the field — otherwise the newly correct parser would have failed its own pre-existing happy-path test.

5. Negative cases as first-class tests. Cover absent token_type, an unknown template placeholder, an unsupported token type such as mac, and a truncated Base64url segment. The cortexfs test plan adds a missing-token_type failure case to the existing hermetic OAuth regression; the synchro fix requires the defect to stop reproducing.

6. A test for the error path itself. Assert that the client-facing error is bounded and contains no parser internals. This is the criterion that would have caught C12 before an audit did.

Negative tests: absent fields, unknown tokens, truncated segments

Negative cases deserve their own emphasis, because they’re where the parser’s boundaries actually live. Positive tests describe what a parser accepts; negative tests describe what it refuses, and refusal is the behavior that protects everything downstream.

Four cases cover most of the risk surface. An absent required field — the token_type case. An unrecognized token in a registry-driven parser — the template placeholder case. A recognized-but-unsupported token — the mac case, where the field is present and well-formed but the implementation doesn’t support the semantics. And a truncated input — a Base64url segment whose padding has been stripped or whose length is wrong. Each should have a named test, and each test should assert on the specific error the caller receives, not just that an exception was raised.

Conclusion

A token parser isn’t one thing. It’s four different parsers sharing a name, and the format you’re decoding decides which one you need, what its output may claim, and where it has to stop.

Name your token format first, then use the decision table to pick the branch. If it’s a JWT or OAuth token, decode it for inspection but verify the signature before you trust it, and reject a missing or unrecognized token_type instead of defaulting to Bearer. If it’s a lexical, template, or design token, define the token registry, build it once, fail loudly on anything unrecognized, and back the result with a regression test that fails without your fix.

FAQ

Is a token parser the same thing as a tokenizer or a lexer?

They overlap, but they aren’t identical. A tokenizer or lexer produces the token stream; a token parser consumes tokens and gives them structural meaning. In practice, plenty of codebases use the terms interchangeably, so read the token format before assuming which one a given library means.

Does a JWT parser verify the token’s signature?

By default, no. Decoding reverses Base64url and deserializes JSON — nothing more. Verification requires the HMAC secret or RSA public key, and a decoded token can carry any claims its sender invented. For anything security-relevant, use libraries with signature verification explicitly enabled — the same decode-only boundary applies to any online JWT parser you use for inspection.

Why does RFC 6749 require token_type, and is it case-sensitive?

RFC 6749 §5.1 requires both access_token and token_type in a successful response so the client knows how to present the token. token_type is case-insensitive: Bearer, bearer, and BEARER are all valid. RFC 6749 §7.1 adds that a client must not use an access token whose type it does not understand — so reject rather than default.

Why shouldn’t a token parser’s error message include its internal token details?

Parser internals leak grammar rules, offsets, and token names, which hands an attacker a map of the input format. The synchro issue #53 records an audit-confirmed finding — C12, verdict CONFIRMED, severity Low — of exactly this kind of leak going out to clients. Return a bounded generic error and keep detailed diagnostics in server-side logs behind a stable correlation ID.

Can I parse template placeholders like {BRANCH} or {SEQ:X} safely, and what about ones I don’t recognise?

Parse against a registry of known tokens rather than a loose regex, and anchor the pattern. Raise on any unrecognized placeholder instead of leaving it in the output — silent passthrough produces duplicate or malformed identifiers. Handle escaping explicitly so injected values containing braces aren’t re-parsed on a later pass.

Should I write my own token parser or use a library?

Use a library when a standard defines the format — JWT and OAuth have mature, audited implementations, and hand-rolling the security path is where bugs live. Write your own when the token set is small, the grammar is yours (template placeholders, a domain-specific syntax), or you need it to fail loudly on unknown tokens in a way no general library does. Either way, build token sets once and cover the negative cases with tests.