Ettin reranker (32M/150M) tops MTEB claims

Hugging Face did publish the Ettin reranker family this week, and the social posts were pointing to a real release rather than an orphaned benchmark claim. A Hugging Face blog post dated May 19 said six new Sentence Transformers cross-encoder rerankers were released: 17M, 32M, 68M, 150M, 400M and 1B parameter variants. (huggingface.co) The company said the models were built on Ettin ModernBERT encoders and released with data and a full training recipe. (huggingface.co) The two models highlighted in social chatter — 32M and 150M — also have live Hugging Face model pages. Those pages list Apache 2.0 licenses, direct download and inference support through Sentence Transformers, and evaluation sections tied to MTEB English v2 Retrieval. (huggingface.co) ### Were the 32M and 150M models actually released, or was this just a benchmark graphic? The answer is release, not tease. Hugging Face’s May 19 blog post names both `cross-encoder/ettin-reranker-32m-v1` and `cross-encoder/ettin-reranker-150m-v1` among the six published models. The post says the release also includes the training recipe and the dataset used to produce the family. (huggingface.co) The model cards for the 32M and 150M versions are live on Hugging Face. Each page identifies the model as a cross-encoder fine-tuned from `jhu-clsp/ettin-encoder-32m` or `jhu-clsp/ettin-encoder-150m`, respectively, using `cross-encoder/ettin-reranker-v1-data`. (huggingface.co) ### What do the MTEB claims actually cover? The model pages say the evaluation was done on the full MTEB English v2 Retrieval benchmark. They also say the setup used MTEB’s two-stage reranking flow, with the reranker paired with six embedding models and reranking the top 100 results. (huggingface.co) That matters because MTEB reranking numbers are not standalone “one model versus another” scores in the same way as a single forward pass benchmark. The MTEB documentation describes a two-stage process: first-stage retrieval saves candidates, then a cross-encoder reranks them, typically with `top_k=100`. (huggingface.co) ### Did Hugging Face itself say Ettin beat larger models? Hugging Face’s own language is narrower than some of the social framing. The May 19 blog post says the six rerankers are “state-of-the-art at their respective sizes,” which is a size-bucket claim rather than a blanket statement that every smaller model beats every larger rival. (huggingface.co) The 32M and 150M model cards direct readers to the release blog for “training recipe, evaluation results, and speed benchmarks against other public rerankers,” and say the evaluation section contains “headline numbers.” But in the material reviewed here, Hugging Face does not appear to make the exact social-media formulation that the 32M and 150M variants broadly “top MTEB” without qualification. (github.com) ### Why are people talking about local deployment and “open-weight sovereignty”? Apache 2.0 is listed on both model cards, and the models are distributed through Hugging Face with standard Sentence Transformers usage instructions. (huggingface.co) That means the weights are available for direct download and local inference, which is the factual basis for the local-deployment argument circulating in posts. The blog post also frames the release as part of open-source and open-science work, and says the full training recipe is included. Claims about “sovereignty” are therefore an inference from the open-weight licensing and downloadable model distribution, not a quoted Hugging Face slogan in the source material reviewed. (huggingface.co) ### So what is solid, and what remains unproven from the social thread? The solid part is straightforward: Hugging Face released the Ettin reranker family on May 19; the 32M and 150M models are live; both were evaluated on MTEB English v2 Retrieval using two-stage reranking; and both are openly licensed under Apache 2.0. (huggingface.co) The unresolved part is the strongest social phrasing about exactly which larger models they beat, under which embedder pairings, and by what margins. The Hugging Face blog says results exist for multiple embedder pairings, but the source material reviewed here does not provide a clean independent leaderboard extract for the specific “32M and 150M top larger models” formulation. (huggingface.co) As of May 21, the most direct place to verify the claim is Hugging Face’s own Ettin release post and the two model cards for `cross-encoder/ettin-reranker-32m-v1` and `cross-encoder/ettin-reranker-150m-v1`, where the evaluation setup and downloadable files are posted. (huggingface.co)