LoRA Speedrun Leaderboard Cuts Qwen2.5-1.5B Fine-Tuning to 6:05 at 61.1% GSM8K Accuracy

LoRA Speedrun has posted a 6-minute, 5-second Qwen2.5-1.5B fine-tuning record that reached 61.1% exact-match accuracy on GSM8K. The result cuts 49% from the project’s 11-minute, 57-second baseline while producing a higher evaluation score.

The improvement did not require a larger GPU, a different base model, or more trainable parameters. Both runs used one Nvidia L40S and the same rank-16 LoRA configuration. The record instead changed how training examples were arranged and which tokens contributed to the loss.

That distinction makes the LoRA Speedrun leaderboard more interesting than a one-off optimization claim. Its fixed task, hardware, accuracy threshold, and verification protocol turn fine-tuning performance into a reproducible systems contest. The central rivalry is no longer one model against another. It is efficient data handling against conventional training pipelines that spend compute on avoidable padding and prompt tokens.

The LoRA Speedrun Leaderboard Changes What Counts as a Win

The headline result combines lower wall-clock time with a higher score under the same published track rules.

Developer Saivineeth posted both the baseline and current record on July 18, 2026. The project’s first track asks competitors to fine-tune Qwen2.5-1.5B on the GSM8K training split using one L40S GPU.

A submission must reach at least 57% exact-match accuracy. Exact match means the extracted final answer must equal the reference answer, rather than merely resembling the expected reasoning.

The baseline trained for three epochs and finished in 11 minutes, 57 seconds. It scored 59.4% on the project’s GSM8K evaluation. The faster submission trained for two epochs, finished in 6 minutes, 5 seconds, and scored 61.1%.

Those figures represent a 5-minute, 52-second reduction. They also show a 1.7 percentage point accuracy increase within the project’s evaluation setup. The time improvement is approximately 49%, which supports the repository’s shorthand description of an almost twofold speedup.

The project publishes its leaderboard, task specification, scripts, and verification artifacts in the speedrun repository. That openness matters because fine-tuning speed claims often combine unlike models, accelerators, datasets, and stopping conditions.

LoRA, or low-rank adaptation, freezes the original model weights and trains smaller matrices attached to selected layers. The original LoRA paper presented this approach as a way to reduce trainable parameters and memory demands.

LoRA Speedrun limits submissions to adapter-only training with no more than 30 million trainable parameters. Competitors can still change adapter placement, rank, schedules, example ordering, kernels, quantization, and stopping behavior.

That freedom creates room for meaningful engineering competition. The constraints prevent a submission from winning simply by replacing the task, adding GPUs, or fine-tuning every model parameter.

The metric is training wall-clock time, not floating-point operations or training steps. Wall-clock measurement captures data loading, padding, kernel behavior, optimizer work, and other costs experienced during an actual run.

The timer does not measure every stage of an end-to-end model project. Evaluation remains outside the training time, and the repository can prefetch model and dataset files. The record therefore describes a controlled training race, not the total duration of production deployment.

Still, the controlled scope is the project’s main advantage. A narrowly defined record can reveal optimization effects that broad, loosely documented comparisons often hide.

This design follows the spirit of modded-nanoGPT, a public competition that fixes a training target and invites faster implementations. LoRA Speedrun applies that format to parameter-efficient adaptation rather than model pretraining.

The record creates a simple question for future submissions: can another technique beat 6:05 while clearing the same accuracy bar three times? That is more testable than claiming a fine-tuning library is generally faster.

Why Data Efficiency Now Pressures Fine-Tuning Frameworks

The result puts pressure on default training pipelines because most of the gain came from changing token utilization, not adapter mathematics.

Many fine-tuning comparisons focus on adapter rank, quantization, optimizer choice, or GPU class. Those variables matter, but they can distract from wasted work inside ordinary batches.

Variable-length examples are commonly padded so every sequence in a batch has the same length. Padding tokens make tensors easier to process, yet they do not add useful training content.

If one example is far longer than its neighbors, the shorter examples receive more padding. The accelerator then processes positions that contribute little or nothing to learning.

Sequence packing addresses that problem by placing multiple examples into a shared fixed-length sequence. Better packing raises the proportion of useful tokens processed during each forward and backward pass.

The winning submission combined packing with completion-only loss masking. This masking calculates training loss only on response tokens while excluding the prompt portion from the learning objective.

The prompt still provides context. However, the optimizer does not spend its objective on teaching the model to reproduce the question text.

Together, these changes target two different forms of inefficiency. Packing reduces unused sequence space, while completion-only masking concentrates the learning signal on the desired output.

The winning run also used two epochs instead of the baseline’s three. An epoch is one pass through the selected training data. Cutting an epoch removes substantial work, provided the model still clears the accuracy threshold.

This is where the 61.1% score becomes essential. A faster run that fell below 57% would not qualify, regardless of how efficiently it processed tokens.

The faster submission did not merely scrape past the threshold. According to its public verification report, the accepted result survived the project’s prescribed reruns and audits.

The finding pressures maintainers of popular fine-tuning stacks to examine their defaults. Convenience-oriented pipelines may favor simple batching and full-sequence loss because those choices work across many datasets.

A speedrun asks a different question. It asks which generic conveniences impose measurable costs once the task and output format become predictable.

That pressure also reaches internal machine learning teams. A team might spend time comparing newer accelerators before measuring padding ratios, useful tokens per second, or unnecessary loss computation.

Hardware upgrades can shorten runs, but they may leave inefficient data preparation untouched. Software gains can sometimes transfer across supported accelerators without changing model quality targets.

The LoRA Speedrun result does not establish a universal 49% reduction. The gain depends on sequence-length distribution, prompt format, batch construction, and the number of epochs actually needed.

Datasets containing uniformly sized examples will offer less packing benefit. Tasks that require modeling every token may not support completion-only masking in the same way.

Even so, the record changes the burden of proof. A fine-tuning pipeline that ignores packing now needs to explain why its workload does not benefit from denser token use.

The same applies to loss design. Teams should identify whether they want the model to learn an entire conversation transcript or only the assistant’s completion.

For developers, these questions affect experimentation speed. Shorter runs allow more schedules, adapter placements, and data selections to be tested within the same compute window.

For organizations, the larger consequence is iteration capacity. The operational value comes from running more controlled experiments, not from celebrating a single six-minute job.

Teams also need records of configurations, datasets, and evaluation decisions. A searchable engineering knowledge base can help preserve why one training recipe replaced another.

Without those records, faster iteration can produce more experiments but less institutional understanding. Speed has value only when teams can reconstruct which change caused an outcome.

Packing and Loss Masking Beat the Plain LoRA Baseline

The primary contest is optimized token handling versus a plain LoRA pipeline that treats convenience as a neutral choice.

The baseline used rank-16 LoRA adapters across all linear layers, three epochs, and a cosine learning-rate schedule. The record retained the same core adapter configuration.

That makes the comparison unusually focused. The winning approach did not switch from LoRA to another parameter-efficient method or reduce the adapter budget.

Sequence packing changes the composition of each model input. Instead of padding every example independently, the pipeline fits multiple training records into longer, more fully occupied sequences.

A correct implementation must preserve example boundaries. It cannot let one problem’s tokens become unintended context for another problem or corrupt labels across packed records.

Completion-only masking changes the target labels. Prompt positions receive an ignored label, while response positions remain active for cross-entropy loss.

Cross-entropy loss measures how much probability the model assigns to the expected next token. Masked positions do not contribute to that calculation or its associated learning signal.

The combination improves useful work per unit of time. Packing feeds fewer empty positions through the model, while masking directs optimization toward answer generation.

The two-epoch schedule contributes another direct reduction. However, fewer epochs alone would not explain the higher reported score without considering batch composition and objective targeting.

The result therefore looks less like a new LoRA algorithm and more like disciplined systems engineering. It extracts more value from the same adapter design by reducing waste around it.

This distinction matters when comparing the record with techniques such as QLoRA, DoRA, PiSSA, rsLoRA, and LoRA+. Those methods alter precision, parameterization, initialization, scaling, or learning rates.

LoRA Speedrun lists several of these methods as open directions rather than completed victories. A future submission might combine them with packing or find that their overhead outweighs their benefits on this short run.

For example, quantization can reduce memory use, but quantization and dequantization also add work. On a 1.5-billion-parameter model that already fits comfortably, memory savings do not automatically produce lower wall time.

Custom kernels present a similar tradeoff. A fused operation can reduce memory movement and launch overhead, but compilation time or shape constraints can complicate short training jobs.

The public race exposes those tradeoffs in a common currency. A method wins only if its full training path finishes faster and still meets the evaluation requirement.

That focus separates research novelty from operational usefulness. A method can be scientifically interesting without improving this track, while a straightforward batching change can dominate the leaderboard.

The Qwen base model also shapes the contest. The official Qwen model card describes a 1.54-billion-parameter causal language model with a 32,768-token context length.

The speedrun does not test that full context length. Its configuration, task formatting, and batch limits determine the actual sequence workload.

GSM8K contains grade-school mathematics word problems that require multi-step reasoning. The original GSM8K dataset includes 7,473 training problems and 1,319 test problems.

The benchmark remains useful because answers can be evaluated automatically. However, one exact-match score does not measure broad mathematical reasoning, reliability, or behavior on unfamiliar domains.

The project acknowledges another concern: modern base models may have encountered similar mathematical content during pretraining. The leaderboard treats 57% as an optimization target, not evidence of newly discovered reasoning ability.

That framing is important. The record should not be compared directly with every published Qwen2.5-1.5B benchmark score.

Prompt templates, decoding rules, answer extraction, model variants, and evaluation harnesses can alter GSM8K results. Qwen2.5-1.5B and Qwen2.5-1.5B-Instruct are also distinct checkpoints.

The valid comparison is internal. Under the project’s frozen track, the packed two-epoch submission finished faster and scored higher than its published baseline.

A broader claim requires replication on other model families and tasks. That requirement leads directly to the project’s second track.

The 61.1% Result Has a Deliberately Narrow Meaning

The leaderboard offers stronger verification than a screenshot, but its current evidence does not establish a universal fine-tuning recipe.

Each candidate record begins as a pull request containing code, configuration details, notes, and self-reported results. Continuous integration then performs static validation.

The repository says an automated security review checks for network access, test-set contact, harness tampering, and data-exfiltration attempts. A maintainer must approve the submission before official execution.

The verification harness runs approved code three times with fresh seeds. All three runs must clear the target, and the official time is their mean.

Those runs occur inside a network-blocked Modal sandbox on the specified L40S. The harness also audits trainable adapter parameters and checks model and dataset hashes.

The judging protocol is more informative than an isolated benchmark post. It defines acceptance rules and leaves artifacts that other developers can inspect.

Modal documents the L40S as a supported accelerator with 48 GB of GPU memory. Its GPU documentation also notes that fixed accelerator names help users request a specific hardware class.

However, three reruns do not settle every source of variance. Host behavior, software versions, thermal conditions, and cloud scheduling can still affect short wall-clock measurements.

A 6-minute run amplifies the importance of small overheads. Startup behavior, cached artifacts, data-loader timing, and compilation decisions occupy a larger share of total time.

The project addresses some of these concerns by freezing its environment and using identical hardware. Public verification reports also reveal whether a proposed gain survives fresh seeds.

Still, the record belongs to one model-task pair. GSM8K has relatively structured prompts and answers, which can favor completion-focused training.

Other tasks may require the model to reproduce formats across an entire sequence. Dialogue training may include multiple assistant turns, tool calls, system instructions, or preference labels.

Packing also becomes harder when examples use complex attention masks. A naive implementation can permit information leakage across example boundaries or distort positional behavior.

The second track is the project’s first answer to this limitation. It pairs SmolLM2-1.7B with SQuAD v1.1, a question-answering dataset with different input and output characteristics.

As of July 21, 2026, that track lists an 11-minute, 8-second baseline with 77.5% exact match. It uses the first 20,000 SQuAD training examples and one epoch.

The record-setting packing technique has not yet topped both tracks. Therefore, the repository itself distinguishes a track-specific record from a generally transferable method.

That is the most important skeptical reading. The 6:05 result verifies an optimization under frozen conditions, but transfer remains an open empirical question.

The leaderboard also sets a threshold rather than maximizing accuracy. This design rewards the fastest qualifying run, even if a slower configuration would produce a meaningfully better model.

That tradeoff matches the project’s purpose, but production teams face different objectives. They may need safety evaluations, domain coverage, calibration, and regression testing beyond one benchmark.

A medical, legal, or financial adapter should not stop training simply because it crosses a public accuracy floor. Its deployment standard must reflect the consequences of incorrect outputs.

Likewise, exact-match evaluation can hide reasoning defects. A model may reach the correct final number through unstable logic, while a formatting mistake can mark sound reasoning as incorrect.

The 61.1% score should therefore be read as a qualification result inside this race. It is neither a full model evaluation nor proof that Qwen2.5-1.5B gained general mathematical competence.

That narrow interpretation does not diminish the engineering result. It makes the claim precise enough to test.

What the Next LoRA Speedrun Records Need to Prove

The next three signals will show whether 6:05 is a durable systems result or an early track-specific optimization.

The first signal is an independently authored submission that beats 6:05 on the same Qwen2.5-1.5B track. The current leaderboard’s baseline and record share the same author.

The repository’s verification process reduces the risk of an irreproducible claim. However, outside participation would test whether the instructions, environment, and optimization are usable beyond the project creator.

A faster accepted entry would strengthen the speedrun format itself. It would also show that the benchmark has enough optimization headroom to attract competing approaches.

Watch whether new entries use data selection, one-epoch schedules, fused kernels, or alternate LoRA placement. Each strategy targets a different bottleneck and would clarify where the remaining time goes.

The second signal is whether sequence packing and completion-only loss masking improve the SQuAD track. Transfer across SmolLM2-1.7B and a different task would support a broader efficiency claim.

Failure would also be informative. It would suggest that the record depends heavily on GSM8K’s sequence lengths, output format, or training dynamics.

A successful cross-track submission does not need to produce an identical 49% reduction. It needs to show that the technique retains useful speed benefits while meeting SQuAD’s 75.5% threshold.

The third signal is the behavior of official verification times across more submissions and software changes. Stable reruns would strengthen confidence in wall-clock ranking.

Large variation would indicate that the leaderboard needs tighter reporting around kernels, driver versions, caching, or host-level conditions. Short records require unusually careful timing discipline.

These signals matter more than raw GitHub attention. Stars can reveal interest, but they cannot validate a training method’s speed, accuracy, or transferability.

The LoRA Speedrun leaderboard has already supplied a credible first result. It reduced a published baseline from 11:57 to 6:05 while raising GSM8K exact match from 59.4% to 61.1%.

Its deeper contribution is the contest design. Fixed hardware and tasks force algorithmic choices, kernels, data pipelines, and stopping rules into one measurable race.

Developers should resist turning that result into a universal recipe. Instead, they can reproduce the submission, inspect token utilization, and test the same changes on their own workloads.

The most useful question is not whether every LoRA job can finish in six minutes. It is whether current pipelines spend nearly half their time on work that never needed to happen.

Over the next several months, accepted outside records, cross-track transfer, and stable verification runs will provide the answer. Until then, the 6:05 mark is a strong benchmark result with clearly defined boundaries.

If your team fine-tunes smaller language models, measure padding, active loss tokens, and useful tokens per second before changing hardware. Then document each controlled run and compare outcomes under a fixed quality threshold. The next meaningful LoRA Speedrun record will come from evidence that survives those controls, not from a faster stopwatch alone.