Open models

Reading an open-weight model card

A release announcement describes a product. A model repository describes an artifact. A comparison needs the artifact.

By LLM Model Notes · 4 November 2026 · 9 minute read

Chinese model labs publish many of their large releases as open weights on Hugging Face. That makes the repository the most precise public record of what was released. It is also easy to read carelessly. The card, the configuration file, and the license file each answer a different question, and the answers do not always match the headline of the announcement.

Pin the exact artifact

Start with the repository identifier and the commit you downloaded. A repository name such as DeepSeek-V4-Pro is a moving label: the files under it can change after release. Record the commit hash with every measurement, the same way you would record a dataset revision.

Look for separate repositories that share a family name. DeepSeek created the DeepSeek-V4-Pro repository for a preview in April 2026 and a later build as DeepSeek-V4-Pro-0813. Results for one do not transfer to the other. A date suffix, a -preview tag, or a code-specific variant such as Kimi-K2.7-Code marks a different artifact, not a different label for the same one.

Repository dates also need care. The creation date that the hub shows is the upload date of the repository. It can be earlier than the public announcement, when a lab prepares the repository in advance, or later, when a lab first ships an API. When a date matters, cite the source you read it from.

Separate total and active parameters

Most large releases from these labs are mixture-of-experts models. A router sends each token to a small number of expert blocks, so only part of the network runs for any one token. Cards therefore state two numbers, and they answer different questions.

The total count decides how much memory the weights need. The active count is closer to the compute cost of one token. Some names encode both: in the Qwen scheme, 35B-A3B means 35 billion total and 3 billion active. Do not compare a mixture-of-experts model with a dense model by total size alone, and do not compare them by active size alone. The serving note works through the consequences.

Read the context length twice

Cards often give a native context length and a longer extended length. For example, the Qwen/Qwen3.8-27B card states 262,144 tokens natively and extension to about one million. Extension usually depends on a position-scaling method that the serving runtime must enable, and quality at the extended length is a separate claim from support for it.

Check the configuration file as well. The max_position_embeddings field in config.json is what most runtimes read by default. When the card and the configuration disagree, the runtime will follow the configuration unless you override it, so record which value your test used.

Rule of thumb: treat any context figure as a compatibility limit. Measure retrieval or reasoning at the lengths you need before you rely on them.

Open the license file

The license tag at the top of a repository is a summary chosen by the uploader. The license file is the document that applies. Among recent releases the difference matters:

• Many repositories use a standard license such as MIT or Apache-2.0.
• The Kimi-K2.5 and Kimi-K2.6 repositories use a license named "Modified MIT". Read the modification before you assume MIT terms.
• Several flagship releases use a custom license with the model's name, for example the "Kimi K3 License", the "GLM-5.3 License", and the "Qwen3.8-Max License". A smaller model in the same family can still use Apache-2.0.
• Tencent published the Hy3 preview under the "Tencent Hy Community License Agreement", which excludes some territories, and the later Hy3 release under Apache-2.0.

The practical rule is to record the license per repository and per revision, not per model family.

Check what the card does not say

A card is written by the publisher. It usually reports benchmark scores under the publisher's own prompts and settings, which the benchmark note explains how to review. It rarely states training data composition, contamination checks, or the chat template differences between builds. Missing information is not a negative result, but it should appear in the limitations of any comparison that depends on it.

Also check the files the card refers to. The tokenizer files and the chat template decide how your prompts become tokens. Two builds with the same weights and a changed template can behave differently on the same input.

A card review checklist

• Repository identifier and commit hash recorded with every result.
• Preview, dated, and task-specific builds treated as separate artifacts.
• Total and active parameters both noted, with the architecture type.
• Native context, extended context, and the configuration value recorded.
• License file read and its exact name recorded for this revision.
• Tokenizer and chat template revision captured with the prompts.

Primary references

• Hugging Face model cards documentation — what the card fields and metadata mean.
• Model Cards for Model Reporting — the reporting practice that model cards come from.
• Model tracker — recent releases with the facts checked on each repository.