Aspaklarya — Qwen3-VL-8B for remote-sensing VQA + captioning

LoRA finetunes of Qwen3-VL-8B-Instruct, trained jointly on visual question answering and image captioning over overhead imagery. 17 experiments, 17 evaluated recipes. All model repos are private.

Setup

Winners are merged into the bf16 base and re-validated: adapters are evaluated on a 4-bit base, merged weights are not.

Configurations

Constant everywhere: effective batch 32 (except 014 at 16, 015-017 at 64), warmup 0.03, grad clip 0.3 (1.0 for 011/012), seed 3407.

Results

Best checkpoint per run on a fixed 1000-image draw (2000 graded items), with bootstrap standard error. Overall = mean of caption and VQA. In-dom. = the target overhead-imagery subset (491 images); RSICD = the public RSICD remote-sensing benchmark subset (509 images). Sorted by in-domain overall; bold marks the best value in each column.

Every finetune beats both references overall: +17.1 over the base model and +8.7 over the in-house reference on the full set.

Learning rate at large batch

Aspaklarya-015 gave the best captioning in the table at effective batch 64, but it was trained at 2e-4 — the rate tuned for batch 32. Two follow-ups tested whether the batch-64 recipe was simply under-stepped. Scores below come from a separate 500-image draw used for live checkpoint selection, so they compare only with each other.

Linear LR scaling is wrong for LoRA here: 4e-4 peaked at epoch 1 and then lost ground at every later checkpoint, ending below where 2e-4 started. The adapter's effective step size does not follow the full-finetune scaling rule, so raising LR with batch size overshoots rather than compensating.

Error bars

Bootstrap over images, 3000 resamples. Absolute standard error:

Paired model-vs-model error is smaller because image difficulty cancels: 0.42 overall on the full set, 0.56 in-domain. Use the paired figure for any "A beats B" claim.

Caveat

The top eight runs are statistically tied: every confidence interval against the leader straddles zero. VQA error is ~2.3x caption error, so no finetune-vs-finetune VQA difference in any column is resolvable; only caption gaps and the gaps to the two references are real. Doubling the eval set from 500 to 1000 images did not separate the field, and it reshuffled the order.

Findings

• Optimization is saturated. LR, rank, alpha, dropout, weight decay, gradient clipping, batch geometry, effective batch size (16 / 32 / 64) and LR scaling at large batch were all swept. None beat the noise floor.
• The slices disagree. Ranking by in-domain score almost inverts the RSICD ranking: the top three in-domain models place 11th, 2nd and 16th on the general set. The selected model wins by never being bad anywhere, not by leading any single slice.
• Captioning is the real gain. It improves on every slice, by +14 over the base model and +10 to +19 over the in-house reference.
• The in-domain VQA win is not established. The in-house reference posts the single highest in-domain VQA score in the table; it collapses outside that domain, where it falls below the untuned base model.
• The vision tower matters. Freezing it is the one ablation that consistently hurt.
• Extra QA per image does not help. 4-QA and 6-QA mixtures tie with 2-QA.
• Larger batch trades generality for domain fit. Effective batch 64 gives the best in-domain captioning in the table and the worst RSICD VQA.
• Data is the remaining lever, not hyperparameters.