AI Meerkat-7B becomes first small language model to pass US Medical Licensing Exam with a score of 74
Developed by joint research team from Korea University, AIGEN Sciences, and I.C.L.
Leading new business expansion and technological innovation as medical AI available on personal computers

▲ Professor Kang Jae-woo from the Department of Computer Science and Engineering

The research team led by Professor Kang Jaewoo from the Department of Computer Science and Engineering, in collaboration with AIGEN Sciences and Imperial College London (ICL), has successfully developed Meerkat-7B, a small language model (sLLM) containing fewer than 7 billion parameters. It is the first sLLM to pass the United States Medical Licensing Examination (USMLE).

While large language models (LLMs) led by companies such as OpenAI and Google have shown promising results, the risk of sensitive data leakage when using external cloud services makes them unsuitable for use in hospitals or corporations. As a result, there is growing demand for sLLMs that can be installed internally to enhance security, known as "on-premises" solutions.

sLLMs are models that reduce the number of parameters to decrease costs and that enhance accuracy through fine-tuning. While OpenAI's GPT-3.5 (ChatGPT) boasts 175 billion parameters and Google's 'PaLM' reaches 540 billion, Meerkat-7B has just 7 billion. This size allows the model to be installed and utilized on a single PC, giving it a significant advantage.

Meerkat-7B is an sLLM model specialized in the field of life sciences, possessing the multi-step reasoning ability necessary to address complex medical issues. While the average passing score for the United States Medical Licensing Examination is 60, the previous leading sLLM, MediTron-7B, failed with a score of 52. However, Meerkat-7B successfully passed with an impressive score of 74, demonstrating its performance. Additionally, it outperformed the GPT-3.5 (175B) model by an average of 13% across seven medical benchmark evaluations, highlighting the significant progress in open-source model development in the medical field.

Specialized language models like Meerkat-7B have the potential to enhance efficiency in medical and administrative tasks within hospitals, such as by supporting clinical decision-making and organizing non-standardized medical charts. In pharmaceutical companies, they can assist in labor-intensive and specialized tasks such as patent analysis, clinical trial design, and document writing, thereby reducing the workload of professionals in these fields.

Professor Kang Jae-woo from the Department of Computer Science and Engineering stated, "In the field of life sciences, where over 3,000 research papers are published daily, identifying and validating new disease target proteins crucial for drug development is a highly time-consuming task." He added, "We anticipate that Meerkat-7B will significantly improve the efficiency of discovering new drug targets, and, based on this achievement, we are also preparing new business models utilizing LLMs specialized for medical science."

Furthermore, Professor Kang Jae-woo has founded AIGEN Sciences, an AI-based drug development company, and is currently undertaking 14 proprietary drug development projects targeting cancers and rare diseases. AIGEN Sciences is expected to play a vital role in driving innovation in the pharmaceutical and medical industries through AI.

<Fig 1>

▲ Performance comparison between Meerkat-7B and existing open-source language models. With fewer than 7 billion parameters. 

Meerkat-7B achieves a score of 74 and becomes the first open-source small language model to surpass the pass mark (60 points) of the United States Medical Licensing Examination (USMLE).