ServiceNow Launches BigCode Project, Open Source Generating AI Program

The BigCode project was created as an AI system to develop open source (photo: ServiceNow)

JAKARTA - ServiceNow Research and Hugging Face announced a project to develop an Artificial Intelligence (AI) system called BigCode. The project aims to develop state-of-the-art Large Language Models (LLM) with open and responsible code on Monday, September 26.

This LLM code will later work in various domains, tasks, and programming languages. These models can help professional developers and citizens alike with coding new applications.

The BigCode project will be open to anyone with an AI research background, and invites them to collaborate on the following topics:

We're excited to announce our collaboration with @huggingface to develop state-of-the-art LLMs for code. Code LLMs enable the completion & synthesis of code & work across a wide range of domains, tasks, & programming languages. #BigCodeProject

Read more: https://t.co/C1XFgr3nkf https://t.co/PhWprKKtwD

— ServiceNow Research (@ServiceNowRSRCH) September 26, 2022

• Representative evaluation suite for LLM code covering a wide variety of tasks and programming languages 
• Responsible dataset development and governance for LLM code
• Faster training and inference methods for LLM Representative evaluation suite for LLM code covering a wide variety of tasks and programming languages

"BigCode's first goal was to develop and release a data set large enough to train sophisticated code language models. We will ensure that only files from repositories with permissive licenses enter the data set," the company wrote on the ServiceNow blog.

With that data set, they will then train a language model with 15 billion parameters to code, using ServiceNow's in-house GPU cluster. Furthermore, with the adapted version of Megatron-LM, the company will train the LLM on a distributed infrastructure.

"After the model is trained, we will evaluate its capabilities. Academic research usually stops after evaluation, this is where the work for application practitioners begins," he added.

Throughout the LLM development process, ServiceNow and Hugging Face will uphold the principles of openness and transparency. They admit that these experiments can be expensive and take a lot of time to run, however, the company asked for feedback from participants before it officially started.

"AI practitioners from various backgrounds are invited to join the BigCode project. This invitation is open to those who have a professional AI research background and can make time for the project," the company said.