ControlNet and StarCoder: Roblox research advancements for Generative AI

With generative AI, computers detect the underlying pattern related to the input and produce similar content. This is in contrast to most other AI techniques where the AI model attempts to solve a problem which has a single answer (e.g. a classification or prediction problem). At its annual developer conference, the company recently announced its plans to offer avatar-based voice calls, as part of its broader initiative to cater to users 17 and up. During the conference’s keynote, Roblox CEO David Baszucki also suggested that thousands of adults would eventually meet and form real-life relationships through Roblox over the next five years. It built its initial following with children, and over half of its active users were under 13 as late as 2021. However, adults and professional developers have taken a greater interest in the platform, and now users age 13 and older make up about 57% of its daily active user base of 66 million.

It also makes training a network for a specific task more efficient and thus practical to deploy at scale for our millions of creators. In experiments, ControlNet provides up to a 10x efficiency gain compared to alternative scenarios that require a model to be fully re-trained. This efficiency is critical, as the process of creating new models is time consuming and resource-intensive relative to traditional software development. Making training more efficient conserves electricity, reduces costs, and increases the rate at which new functionality can be added. Making it easier for creators to build games is part of its mission, and Roblox provides creators with a platform that enables end-to-end tools, services and support to help them build the most immersive 3D experiences, Sturman said.

Tests for the new tools, which can make assets and complete code, will roll out in the ‘coming weeks.’

These new AI systems have significant expressive power, but that power is directed largely through “prompt engineering.” Doing so means simply changing the input text, similar to refining a search engine query if it didn’t return what you expected. While this may be an engaging way to play with a new technology such as an undirected Yakov Livshits chatbot, it is not an efficient or effective way to create content. Creators instead need power tools that they can leverage effectively through active control rather than guesswork. Code Assist looks promising, but Roblox is careful to state it's imperfect, and may generate "incorrect" or "misleading" information.

Analytics Insight® is an influential platform dedicated to insights, trends, and opinion from the world of data-driven technologies. It monitors developments, recognition, and achievements made by Artificial Intelligence, Big Data and Analytics companies across the globe. Analytics Insight is an influential platform dedicated to insights, trends, and opinions from the world of data-driven technologies. Generative AI content cannot be submitted to the Illustrative Editorial Collection (IEC) which is for editorial use only. IEC submissions must be accurate portrayals of brands in an authentic, accurate, and unmodified manner as they are found in the real physical world.

The Code Assist and Material Generator betas hope to make game creation less tedious.

“We envision the community as a force multiplier for generative AI, creating an ecosystem that our creators and users can leverage to create content and tools more effectively,” he said. Roblox is standing by the notion that it is building a platform that will enable every user to be a creator – not just those comfortable with Roblox Studio and other 3D content creation tools. Even more powerful, the convergence of media supported by generative AI will allow creators to develop integrated 3D objects that come with behavior built in. For example, a creator could design a car through a simple statement such as “A red, two seater, convertible sports car with front-wheel drive”.

Yakov Livshits
Founder of the DevEducation project
A prolific businessman and investor, and the founder of several large companies in Israel, the USA and the UAE, Yakov’s corporation comprises over 2,000 employees all over the world. He graduated from the University of Oxford in the UK and Technion in Israel, before moving on to study complex systems science at NECSI in the USA. Yakov has a Masters in Software Development.

• And we’ll see the [experiences] that are created by developers become more rich and dynamic.
• Other applications also involve privacy concerns and might affect the area of medical imaging and health-related applications.
• That said, these generative AI tools could take away work from Roblox’s developer and creator community, and I asked Sturman what he would say to people who are worried about that.
• Earlier this year, we shared our vision for generative artificial intelligence (AI) on Roblox and the intuitive new tools that will enable every user to become a creator.

This new system, which detects policy-violating speech directly from audio, is significantly more compute efficient than a traditional ASR system, which will make it much easier to scale as we continue to reimagine how people come together. If the generative AI content was not based on a real person, but it visually appears to resemble a person, then you must click the "People and Property are fictional" box in the contributor portal to certify the people and/or property are fictional. If there are no recognizable people or property in the image, no release is required and you may leave the "People and Property are fictional" box unchecked. You must have all the necessary rights to submit generative AI images to Adobe Stock for licensing as described in our Contributor Terms (e.g., broad commercial use). As with any other tools you use to create image submissions, you must closely review the terms of any generative AI tools that you use to confirm that you have the right to submit generative AI images created with such tools to Adobe Stock for licensing.

First Generative AI Game Creation Tools

We know people sometimes violate our policies unintentionally and we want to understand if an occasional reminder might help prevent further offenses. To help with this, we are experimenting with real-time user feedback through notifications. This works well for companies using it at a smaller scale, but as we explored using this same ASR process to moderate voice communication, we quickly realized that it’s difficult and inefficient at our scale.

However, these off-the-shelf AI systems are not integrated with our platform and they often do not produce “Roblox ready” output that requires substantial follow on work from a creator. We see an incredible opportunity to build generative AI tools Yakov Livshits and APIs focused on Roblox. Roblox Assistant was partly inspired by evidence of users already employing third-party generative AI chatbots like ChatGPT. The difference is that Roblox Assistant is optimized for Roblox’s domain-specific needs.

Generative AI leverages AI and machine learning algorithms to enable machines to generate artificial content such as text, images, audio and video content based on its training data. As you can see above most Big Tech firms are either Yakov Livshits building their own generative AI solutions or investing in companies building large language models. Text Generation involves using machine learning models to generate new text based on patterns learned from existing text data.