Inside IBM Think 2024: IBM Reboots the Future of Hybrid AI
IBM has a brand disadvantage that is seemingly incongruent with the brave new world of generative AI. Many of us have benefited from IBM’s Red Books throughout our careers, but the sleeping giant has often missed the bus on leveraging its immense research. There you go, I said it because I wanted to get the 800 lb gorilla out of the picture.
What you are about to read will make you think again about the reinvigorated IBM that is miles ahead of its legacy perception. It is no longer your dad’s IBM!
My views stem from the recently concluded IBM Think conference in May 2024 in Boston. The rapid announcements from the conference were like drinking from a firehose. I have captured some of the essential developments and their impacts here that I think are the most informative.
So, let’s get down to what are the most important key findings from this event:
- IBM has a laser focus on becoming the go-to hybrid, cross-cloud software infrastructure plane in the data analytics space. They are infusing this strategy with AI and automation, and are using open standards where possible.
- Its offerings are designed to reduce consumer friction and are purpose-built to meet clients’ needs rather than be a general purpose provider of software capabilities.
- It is heavily leveraging its vast ecosystem of partners and its consulting services rather than relying on only in-house products.
You will find how these principles are being applied as we delve into the various new announcements in this blog. To do so, we will examine new developments based on the figure below.
Let’s start with the hottest topic for the last two years — generative AI and models.
AI Models
IBM’s Granite is a family of open source and commercial models. The open-source collection is released under Apache 2.0 license and includes 18 models across four categories: code, language, time-series, and geospatial. The commercial collection expands to a family of custom Granite models that supports beyond English and the custom code models delivered through watsonx Code Assistants (see below). The open source and commercial collection range in size from 3B to 34B parameters.
A hurdle in building large language models (LLM) is that it is exorbitantly expensive, requiring an army of supply-constrained GPUs. LLM providers do not have a technology moat. Their only moat is vast amounts of capital and a deep bench of highly-skilled resources. IBM has wisely decided not to enter this arms race and is concentrating on purpose-built models that serve the needs of their portfolio and their customers.
Take the example of watsonx Code Assistant (WCA) whose Granite code models have been aligned for the following:
- WCA for Z: This product provides end to end capabilities for the mainframe application lifecycle. In this case, the Granite code model can be used to both explain COBOL code and selectively convert COBOL to Java code. It is not a general purpose code translation model, but has been optimized for mainframe application and modernization use cases. IBM intends to continue training the model with additional programming languages, like PL/1, to meet mainframe customer’s needs.
- WCA for Enterprise Java Applications: This upcoming product will provide end-to-end capabilities for the Java application lifecycle, including code generation, test case generation, code explanation, and more. It will also help customers address Java modernization projects, allowing them to use the trained Granite code model to upgrade from older Java versions to newer ones, and modernize legacy Java code to newer runtimes, such as moving from traditional WebSphere to Liberty.
- WCA for Red Hat Ansible LightSpeed: This product helps to accelerate IT automation, using the trained Granite code model to address the creation and explanation of Ansible content from natural language.
We know LLMs suffer from hallucination and low accuracy problems. This is where IBM’s InstructLab methodology comes into the play. It is a model alignment technique for incremental improvements to base models using the powers of taxonomy, synthetic data, and validation to substantially improve accuracy. It has been developed in collaboration with Red Hat.
For example, an experimental Granite code model for Z was trained on foundational data comprising several COBOL books, mainframe information, and Java code. Then, using InstructLab, it was fed hundreds of coding examples which led to the creation of hundreds of thousands of synthetic data elements. This approach resulted in a 20-point accuracy gain of the LLM compared to the prior approach.
IBM supports a host of third party models. At the Think 24 keynote, a speaker from Saudi Authority for Data and Artificial Intelligence (SDAIA) showcased the availability of a new open Arabic LLM called ALLaM available for enterprise adoption through watsonx. IBM also announced a partnership with the French startup, Mistral AI to offer commercial Mistral models (Mistral Large and Mistral Small) in addition to the open-source Mistral models available in watsonx.
The word indemnify has been very near and dear to IBM. Surprisingly, indemnification clauses are not as prevalent in other vendor conferences. As expected, IBM indemnifies not just its own models but even 3rd party ones like from Mistral.
IBM is also easing the development and deployment of these models which is the topic for the next category.
Code Development
Far more value will be created in the application space than on the infrastructure side in generative AI. Although, GPU manufacturers, LLM providers and cloud hyperscalers are raking in the money, for now, the pendulum will shift to AI assistants, agents, and applications. IBM is gearing up to be the platform to develop and deploy these apps at scale and anywhere.
While IBM supports all the best practices for prompt engineering/chain of thought and retrieval augmented generation (RAG), it sees a big future in the creation of small language models (SLMs). In fact, it’s putting its weight behind InstructLab to accelerate the pace of developing SLMs.
The watsonx Code Assistant (WCA), based on the 20B parameter Granite code model, is used for code generation, conversion, and modernizations but also for explaining and documenting legacy code. As COBOL developers retire and skills become sparse, Code Assistant for Z can be used to bridge knowledge gaps. In the case of Code Assistant for Ansible many of the automation developers do not spend all of their time writing automation, so similarly WCA can be used to bridge knowledge gaps as they develop playbooks.
The use of business intelligence (BI) tools has plateaued. Enter IBM’s watsonx BI assistant (in preview) designed to interact with the underlying data stores in natural language by voice or text. It uses semantic automation and reasoning to infer relationships and business significance of results, and also integrates with metric layers from dbt Labs and Cognos.
Data Management
The great Apache Iceberg revolution is upon us and IBM has also caught onto the lakehouse fever. Its lakehouse offering is called watsonx.data. To understand this better, let’s rewind to Think 2022 when IBM made an announcement of its partnership with Amazon Web Services (AWS). It was a tacit acknowledgement of the fact that IBM Cloud was not going to be able to compete with the hyperscalers like AWS, Microsoft Azure, and Google Cloud. This is the wonderful pragmatism IBM is demonstrating these days.
So, what does this have to do with watsonx.data? I’m so glad you asked.
watsonx.data allows the source files to be on Amazon S3 (GA), on-premises or in IBM Cloud. In the future, it will support Azure Data Lake Service (ADLS) and Google Cloud Storage (GCS). The entire lakehouse is built on open standards and includes Apache Parquet file format, Apache Iceberg table format, Presto query engine, and Milvus embedded vector store. IBM introduced an optimized version of Presto called Presto C++. This version was developed by Meta and IBM and it incorporates Velox, an open source C++ acceleration library.
Anyone who owns the data and metadata will hold the keys to the kingdom. IBM is unifying all its data elements accessible through the Iceberg table format by watsonx. This includes operational data in mainframes. At Think 24, IBM announced Data Gate for watsonx which cost-effectively synchronizes db2 for z/OS, IMS for z/OS, and VSAM data on mainframes into Iceberg format. This is another example of the “zero-ETL’’ concept that AWS has been steadily introducing in its stack. Finally, IBM Knowledge Catalog uses LLMs to create a shared semantic layer embedded into watsonx.data.
IBM’s foray into data sharing is via its Data Product Hub. This offering becomes the place to share and discover data products. It is not a marketplace as IBM doesn’t, as yet, provide any transaction capabilities.
We are also seeing a revival of IBM’s vast collection of products, like DataStage, Cognos BI tool, db2 and Informix databases, and Guardium (more on this later). DataStage is IBM’s data transformation tool which is used within its Data Fabric offering.
Governance
AI governance is one of the most important focus areas for IBM. The product that delivers this is watsonx.governance.
AI governance is one of the most important focus areas for IBM. IBM built watonx.governance to help clients build responsible and transparent AI, putting governance in the heart of the gen AI lifecycle. Its three-pillar approach focuses on lifecycle governance, risk management and compliance.
They announce the ability to govern models where they are including deep integrations with Amazon SageMaker. watsonx.governance and Amazon SageMaker will help clients streamline workflows, accelerate time to market for AI initiatives, and manage AI across complex IT environments and ecosystems. They will be able to configure and track fully customizable risk assessment and model approval workflows across multiple stakeholders, providing an audit trail in both watsonx and Amazon SageMaker.
By now, you may have noticed two things — watsonx is all in lowercase, and its name harkens back to the erstwhile Watson family of products. In a candid analyst session, Arvind Krishna, the CEO and Chairman of IBM, shared that while watsonx is completely written from ground-up, IBM didn’t think it practical and cost-effective to go for a brand new name. Hence, it chose to rebrand its previous offering.
To continue IBM’s pragmatic story, it is now leveraging its mature data security product called Guardium for AI security. It is refreshing to see that IBM is using its existing intellectual property (IP) while augmenting them with generative AI capabilities. In its newest incarnation, IBM Guardium is being used to discover which AI models an organization is using and how they are being used. Currently, it is able to automatically discover and monitor models like OpenAI’s GPT family of models, Granite etc. but will add more models in the near future.
At the event, IBM announced an expansion of their partnership with Palo Alto Networks, providing them with IP and consulting services centered around cybersecurity operations, such as threat protection, while IBM focuses its security investment and innovation in identity and access management, and data security, underscoring their focus on hybrid cloud environments and AI models and applications. However, IBM is happy to let its newest partner handle cybersecurity because IBM’s focus is only on data security. Yet another demonstration of IBM’s sharp go-to-market execution.
Hybrid
Red Hat OpenShift is the backbone to IBM’s hybrid offering.It helps companies build out an end-to-end private cloud data and AI story for customers that maintain data on-premises due to data privacy and security reasons or regulatory requirements.
As with the rest of the stack, Red Hat OpenShift has been infused with AI across its three layers:
- RHEL AI — this is the Red Hat Enterprise Linux as the bootable container containing Granite models and InstructLab.
- Red Hat OpenShift AI — an optimized container that can be deployed anywhere — on-premises or in public clouds. The Red Hat team is enabling AI to hybrid cloud.
- Watsonx.ai — which consists of IBM, open-source, and third-party foundation models, AI tools, and governance.
Exceeding even IBM’s own expectations, the mainframe business has grown steadily. It is safe to say that the reports of demise of mainframes are greatly exaggerated. The showfloor at the event had a sleek new Z16 mainframe on display.
Talking about hardware, IBM is also avoiding another race to the ground of building its own GPU chips. While it has in-house GPU AI accelerators, it is not focussed on hardware. In fact, the only hardware area where IBM sees leadership opportunity is in Quantum architecture.
In fact, its Guardium data and AI security offering is already being equipped to handle quantum security. It launched the private preview of Quantum Safe. IBM knows that this technology will not become mainstream for a while but it is taking the necessary steps to own the space when it becomes production-ready.
Automation
Anchoring IBM’s three pillars are hybrid, AI, and automation, which will help the company to achieve its goal of making consumption frictionless. Hence, its acquisition of HashiCorp which provides Terraform and Vault offerings. Coupled with Red Hat’s Ansible, IBM is introducing automation across its entire suite.
Terraform is the industry standard for infrastructure provisioning, while Vault helps with security automation. As generative AI makes hybrid multi-cloud infrastructure complex, HashiCorp can help IBM manage these complexities.
Which brings us to IBM Concert,a new watsonx AI offering that provides a single pane of glass to identify, predict, and remediate anomalies across clients’ source repositories, CI/CD pipelines, and cloud infrastructure. For now, it displays critical vulnerabilities in an intuitive user interface. In the future, it will be the observability UI for cost, compliance, and performance monitoring. It will also be the unified front-end for many of IBM’s assets like Apptio for monitoring technology spend, Turbonomics for performance optimization, and Instana for application performance management.
Finally, the last piece of the watsonx brand is watsonx Orchestrate which leverages generative AI to automate repetitive tasks and streamline workflows. It acts like a digital worker. IBM also announced a capability in watsonx Orchestrate that allows clients to build their own AI Assistants across domains.
Consulting and Partners
Why would I mention consulting services within this technical overview? It turns out that IBM’s in-house consulting services is a huge advantage since many of its peers in the industry do not have such a large organization. AI assistants are still in early stages of maturity and companies are figuring out appropriate use cases. Personalization and customization are therefore critical for gen AI success. Consequently, consulting is IBM’s unique advantage in this race to build out AI assistants.
An interesting concept of “consulting as a code”, called Consulting Advantage, has been launched to help its 160K consultants to develop assistants using watsonx. Over 2,000 assistants have already been developed and are available on IBM’s marketplace.
AI Alliance is another example of the close cooperation between IBM and almost 100 other companies, like Databricks, Meta, Oracle, AMD, academia, etc. This alliance is building tools, safety and ethics policies, and strategies pertaining to AI. Its members include creators of the StarCoder2 LLM, which has been trained on 619 languages and includes a 3B parameter model trained by ServiceNow, a 7B parameter model trained by Hugging Face, and a 15B parameter model trained by NVIDIA.
Finally, IBM has cultivated an ecosystem of over 10,000 partners. A few thousand of them were on hand at Think 24 embracing all the new announcements and showcasing their offerings. Among many of them, IBM showcased its AWS partnership through the Amazon SageMaker integration with watsonx.governance.
Closing Thoughts
IBM has an impressive research organization that has produced amazing technology but its go-to-market has lagged. Arvind Krishna, in a private session with analysts, reminded us that in 1968 when IBM introduced the hierarchical IMS database, it was the first key-value database, predating the namesake category by decades.
Now it is addressing its go-to-market. Culture starts at the top and permeates down through the organization. Arvind is setting that example. He told us that when people make mistakes and are chastised in public, they stop taking risks. According to him, “velocity doubles when one is taking risks.”
The IBM leadership kept reminding us that they feel the market is so big that IBM will thrive along with its peers. Sure enough, while we were attending IBM Think 24, several major conferences were also happening simultaneously, like Dell Technology World and Microsoft Build. Each conference was rife with new announcements.
AI is causing massive disruptions — not just in technology but also in top management. After we saw Snowflake and AWS replace their executives, Samsung followed suit as its hardware is lagging others in the high memory bandwidth chips category.
Microsoft reignited the PC business with the launch of Copilot+PC. And, the staggering funding of AI companies continued unabated. Scale AI, a data labeling company, raised $1B Series F and the French startup H raised $220M seed round. We also saw a few mergers and acquisitions happen during the week with Snowflake buying an AI observability company called TruEra.
As always, I want to thank you for taking the time to hear my thoughts. Your support is what makes me excited to capture the zeitgeist of this moment of data and AI history. If you are interested in watching commentary on IBM Think 24 and the latest in AI, please watch (and subscribe) to the latest episode of the ‘It Depends’ podcast with AI futurist, Bernard Marr, listen to one of the three SiliconAngle theCUBE videos that happened at the event — with John Furrier, IDC President, Crawford Del Prete, and IBM’s General Manager, Data and AI, Ritika Gunnar.
