Highlights

1. “SambaNova, in collaboration with Together, is excited to present BLOOMChat, a 176 billion parameter multilingual chat large language model (LLM). BLOOMChat is available for research and commercial use cases under a modified version of Apache 2.0, which includes RAIL’s use-based restrictions passed down from BLOOM.
2. BLOOMChat is a new, open, multilingual chat LLM that:
   1. Is trained on SambaNova RDUs (Reconfigurable Dataflow Units)
   2. Achieves a win-rate of 45.25% compared to GPT-4‘s 54.75% across 6 languages in a human preference study.
   3. Is preferred 66% of the time compared to mainstream open-source chat LLMs across 6 languages in a human preference study.
   4. Shows strong performance on WMT translation tasks by leading the results among BLOOM variants and mainstream open-source chat models.
3. Inspired by prior work that instruction tuning in one language can benefit performance in another language in multilingual models, we instruction-tuned BLOOM (176B) with English-focused assistant-style conversation datasets including OIG from OpenChatKit , Dolly 2.0, and OASST1 datasets. Despite only fine-tuning in English, we observed substantial increases in chat quality in non-English languages.
4. While this is still an early study, we hope BLOOMChat serves as a valuable resource for the open-source community and as a stepping stone towards further advancements in the field.
5. BLOOMChat is available now [Chat with me!] for a limited time for live chatting on HuggingFace (model hosting and frontend UI provided by Together)…”