Why Salesforce is open sourcing the AI technology behind Einstein

Why Salesforce is open sourcing the AI technology behind Einstein

The CRM vendor is open sourcing the machine learning technology behind its Einstein platform

Credit: Salesforce

Salesforce is open sourcing the machine learning technology behind its Einstein AI platform.

Branded TransmogrifAI, the AutoML library is less than 10 lines of Scala code written on top of Apache Spark, and can be used by developers looking to train machine learning models to predict customer behaviour without having to use a large data set for training.

© Salesforce

"It was developed with a focus on accelerating machine learning developer productivity through machine learning automation, and an API that enforces compile-time type-safety, modularity, and reuse," it states on its website.

"Through automation, it achieves accuracies close to hand-tuned models with almost 100x reduction in time."

In a lengthy Medium post last week, Shubha Nabar, senior director of data science on the Salesforce Einstein team wrote: "Three years ago when we set out to build machine learning capabilities into the Salesforce platform, we learned that building enterprise-scale machine learning systems is even harder."

The key for Salesforce when it was developing Einstein was to be able to deliver smart insights and recommended actions without pooling all of their customer's data together.

This posed quite a serious challenge for the vendor before it acquired a bunch of machine learning specialist companies, including MetaMind and its founder Richard Socher, who is now chief scientist at Salesforce.

"Up to that point, if you couldn’t see or normalise the data, you can’t apply the intelligence," Salesforce CEO Marc Benioff said.

"We have massive amounts of data, petabytes and petabytes, so we have the data that we need and the answer is that we can now operate on that data without interfering with the trust relationship with our customers."

Expanding on this, Nabar wrote: "We have to build customer-specific machine learning models for any given use case.

"Even if we could build global models, it makes absolutely no sense to do so because every customer’s data is unique, with different schemas, different shapes, and different biases introduced by different business processes.

"In order to make machine learning truly work for our customers, we have to build and deploy thousands of personalised machine learning models trained on each individual customer’s data for every single use case.

"The only way to achieve this without hiring an army of data scientists is through automation. Most [AutoML] solutions today are either focused very narrowly on a small piece of the entire machine learning workflow, or are built for unstructured, homogenous data for images, voice and language.

"But we needed a solution that could rapidly produce data-efficient models for heterogeneous structured data at massive scale."

The end product is a way of building a single, modular machine learning model that can function across smaller, more personalised sets of data, giving the impression of multiple, domain-specific models.

Nabar explained: "With just a few lines of code, a data scientist can automate data cleansing, feature engineering, and model selection to arrive at a performant model from which she can explore and iterate further.

"TransmogrifAI has been transformational for us, enabling our data scientists to deploy thousands of models in production with minimal hand tuning and reducing the average turn-around time for training a performant model from weeks to just a couple of hours.

"While this level of automation has been essential for us to scale for enterprise purposes, we believe that every business today has more machine learning use cases than it has data scientists, and automation is key to bringing the power of machine learning within reach."

Explaining why Salesforce decided to bring this project to the open source community, Nabar said: "Salesforce has been a long-time user and contributor to Apache Spark, and we are excited to continue to build TransmogrifAI alongside the community.

"Machine learning has the potential to transform how businesses operate, and we believe that barriers to adoption can only be lowered through an open exchange of ideas and code.

"By working in the open we can bring together diverse perspectives to continue to push the technology forward and make it accessible to everyone."

(Reporting by Scott Carey, Computerworld)

Follow Us

Join the newsletter!


Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags salesforce



The making of an MSSP: a blueprint for growth in NZ

The making of an MSSP: a blueprint for growth in NZ

Partners are actively building out security practices and services to match, yet remain challenged by a lack of guidance in the market. This exclusive Reseller News Roundtable - in association with Sophos - assessed the making of an MSSP, outlining the blueprint for growth and how partners can differentiate in New Zealand.

The making of an MSSP: a blueprint for growth in NZ
Reseller News Platinum Club celebrates leading partners in 2018

Reseller News Platinum Club celebrates leading partners in 2018

The leading players of the New Zealand channel came together to celebrate a year of achievement at the inaugural Reseller News Platinum Club lunch in Auckland. Following the Reseller News Innovation Awards, Platinum Club provides a platform to showcase the top performing partners and start-ups of the past 12 months, with more than ​​50 organisations in the spotlight.​​​

Reseller News Platinum Club celebrates leading partners in 2018
Meet the top performing HP partners in NZ

Meet the top performing HP partners in NZ

HP has honoured its leading partners in New Zealand during 2018, following 12 months of growth through the local channel. Unveiled during the fourth running of the ceremony in Auckland, the awards recognise and celebrate excellence, growth, consistency and engagement of standout Kiwi partners.

Meet the top performing HP partners in NZ
Show Comments