Submit — CS2680 Leaderboard, Assignment 4

CS2680 Leaderboard - Assignment 4

Submit

Name

A label for this submission, shown on your Past submissions page (the leaderboard shows your preferred name, or else your registered name). Up to 64 characters.

Submission: one zip holding src/ and engine/

What goes in the zip

submission.zip
├── src/                 your agent (agentic_loop.py, ...)
└── engine/              your server
    ├── engine.json      the engine and the model (and for other, env)
    ├── launch.sh        starts the server
    ├── flags.txt        optional: the framework's flags
    ├── sglang.patch     optional (or vllm.patch)
    └── files/           optional: text files

src/ and engine/ sit at the top of the zip, or both inside one folder. Anything else in the zip is not graded.
engine.json has three keys, "format", "engine" and "model", and for other a fourth, "env". "format" is 1. "engine" is sglang, vllm or other. "model" is the course's model, a course catalog entry <org>/<name>@<40-hex revision> (the catalog has nvidia/Qwen3.6-35B-A3B-NVFP4@1355db6a052410cfd62085d94b58866fd0f2c3c5). Assignment 4 serves only this model, so the site refuses a folder of your own; serving another model is part of Assignment 5. "env" is the path of your environment folder (Your own engine), which is copied when your grading starts; sglang and vllm may not have it.
launch.sh is run with bash in place of serve.sh or your own batch script, inside the course's sandbox, where $A4 and ~/a4 are not there; #SBATCH lines in it have no effect. It must have Unix line endings. Your server has 45 minutes from the start of the grading job to answer; the end-to-end time starts when it is ready.
flags.txt, for sglang and vllm, may not set the model path, the served model name, the address, the key or the number of GPUs, and may not use --config. Every other flag is yours to set, including those that change the numerics (the KV cache dtype, for example). A flag that drops, prunes or re-quantizes the model's weights serves another model, which is part of Assignment 5.
The patch is git diff --no-renames output against the release the grading uses, and must match "engine": for SGLang only files under python/sglang/ (not python/sglang/kernels/aot/), for vLLM only files under vllm/. No binary changes, symlinks or submodules.
engine/ holds nothing else (no install lists, lock files, wheels or weights: sglang and vllm run from the course's environment, whose source you change with a patch, and other from your environment folder). Every file in it is UTF-8 text.
Sizes: the zip at most 50 MB; engine/ at most 2 MB, with launch.sh at most 64 KB, flags.txt 16 KB, the patch 1 MB, and files/ at most 32 files and 256 KB.

The upload is checked against these rules, and every problem is listed under the Submit button. A zip that is refused is not queued and does not count. Engine conventions lists what launch.sh gets, every flag and patch rule, using vLLM, your own engine, and how to test engine/ on the cluster before you submit; Part 4 of the Assignment 4 page describes the submission.

Drop your zip here, or click to choose one

Any file name, as long as it is a zip archive. Up to 50 MB.

You can have one submission waiting or being graded at a time (cancel it on Past submissions), and 3 gradings a day, midnight to midnight Eastern Time (EST/EDT). A grading counts toward these once its job has a GPU node, and so does one you cancel after that, though a cancelled submission does not count on the leaderboard. Waiting for a GPU is waiting, not grading: a submission waits in the queue, then for a GPU node (which can take a while when the course's GPUs are busy), and cancelling it at any point before its node comes up does not count. Neither does a grading that fails the course's engine check or stops for a problem on the course's side.