Sesame, the startup behind the viral virtual assistant Maya, releases its base AI model

AI firm Sesame unveils CSM-1B, powering virtual assistant Maya.

: AI company Sesame has released CSM-1B, a 1 billion parameter model behind virtual assistant Maya. This model employs residual vector quantization (RVQ) and has some multilingual capabilities. Co-founded by Oculus co-creator Brendan Iribe, Sesame lacks stringent safeguards against misuse. The firm is supported by Andreessen Horowitz, Spark Capital, and Matrix Partners and is also developing AI-enhanced glasses.

In an announcement, AI company Sesame introduced CSM-1B, the foundational AI model behind its viral virtual assistant, Maya. The model boasts an impressive 1 billion parameters, highlighting its substantial computational capacity. CSM-1B employs residual vector quantization (RVQ), a critical technology for encoding audio into discrete units, applied by significant players like Google and Meta in the realm of audio AI. According to Sesame's declaration on the AI development platform Hugging Face, the model is available for commercial use under the Apache 2.0 license.

Sesame has harnessed technology from the Llama family, developed by technology giant Meta, to support CSM-1B, integrating it with a sophisticated audio decoder. This model's foundation is powerful enough to produce various vocal outputs, although current use cautions indicate it hasn't been tailored for individual voices or specific speech scenarios. There seems to be some ability for non-English language outputs due to unintentional inclusions in training data, though results are unreliable.

An area of concern remains the model's minimal inherent safeguards. An open-source nature means developers are entrusted, under an honor system, to avoid using the model for unethical purposes. This includes voice replication without consent or creating deceptive content. When tested, it demonstrated ease in generating speech on sensitive issues, leading to warnings from institutions like Consumer Reports about potential abuse scenarios in AI-generated voice technologies.

Founded by Brendan Iribe, co-creator of Oculus, Sesame emerged as a prominent player in the AI assistant sector when it captured public attention in February. Its technology closely mimics human-like speech patterns, tackling phenomena like the uncanny valley. Maya, along with Sesame’s other assistant, Miles, emulates natural speech pauses and can be conversationally interrupted, sharing capabilities with OpenAI's Voice Mode.

The company has attracted notable venture capital, securing funds from Andreessen Horowitz, Spark Capital, and Matrix Partners. Sesame looks beyond voice technology and ventures into the development of AI-powered glasses, presenting a futuristic vision of wearable tech to potentially integrate seamlessly with daily life.

Sources: TechCrunch, VentureBeat, Hugging Face

technologyAIstartups