Harness GPU Efficiency
At CPU Scale

"Accessibility is the Key".

Long time ago when we launched Sun and Java, there was a new level of accessibility that came forth to the market. HTML was primarily static. However, we added Java applets and applications went onto the internet. The applet changed the way the internet operated. Many programmers across the globe embraced Java and it was easier to write code which before was genuienly object oriented. And most importantly, it ran anywhere , write once run anywhere, no dependencies on processor architecture, memory architecture and so on and as you can see java enabled accessibility across the globe as a result of what we see today, the world wide web that is animated and scalable and safe. We are now at that stage where AI has just started, some architectures have emerged and new hardware to execute them is being designed.Lots of innovation. Kompact’s goal of running efficient and accurate AI on a vast majority of CPUs is not just a technical feat, but a potential catalyst for democratizing AI development. Like Java, it can enable millions of developers to develop test models and use them in production environments, empowering them to contribute to this really cool AI revolution. I want to express my heartfelt gratitude to the entire Xero Labs team for the incredible efforts leading up to this launch. Let the revolution begin. Thank you all.

“Kompact AI type of solutions, very very important to democratize AI and provide quality solutions which are affordable, accessible and also sustainable.“

Today marks a very important event in the AI era. Ziroh Labs, a start-up company based out of Bengaluru, has launched Kompact AI, which will be a new approach to inferencing without GPUs and using conventional CPUs. The most important aspect of this product is that it can be scaled depending upon the need. For example, if you have a language model that you want to build for, say, 50 crore parameters, you can purchase a conventional CPU hardware of around 1.5 lakh and do the inferencing part of it. Of course, the training must be done in a GPU environment, but the inferencing can be happening here. Similarly, if you want to do a 700 crore parameter and probably a conventional server of 8 to 10 lakhs is sufficient. Like this, as we move from, you know, smaller language models as we scale up, we can also scale up the hardware and this is one of the most important contributions of Kompact AI. IIT Madras and the IIT Madras Pravartak Technology Foundation, a not-for-profit foundation of IIT Madras, were involved in fine-tuning and validating specifically from a performance and power consumption point of view, this entire model. It was today released and demonstrated to the public. I am sure this will have multiple use cases. Specific use case, as an example we can give, suppose the government decides to provide an AI assistance for, say, history students across classes. They can train the model at a central place, hiring GPUs. A trained model can be put in the schools using conventional servers with Kompact AI software layer and kids can use. So, for the inferencing part, we do not need GPUs. It is a very cost effective solution. Very importantly, this solution, in our opinion, will be a befitting answer for what we may term as the AI divide. Like digital divide was actually splitting a sector of population which has access to the internet and those who do not. A same type of an AI divide, where people who have lot of money, lot of energy will be having an AI solution, while others who cannot afford that will not have. Such type of a situation is something coming up and it has been discussed in forums including the United Nations. Kompact AI type of solutions, very very important to democratize AI and provide quality solutions which are affordable, accessible and also sustainable. I wish Ziroh Labs all success and this solution is not just for India, not just for our MSMEs, it can even scale to large industries and it is also for the entire globe. I take this opportunity to request some of the big companies, PSUs and the government to encourage this type of a deployment of AI so that we could have a very sustainable AI prevalence in the country. Jai Hind.

"Kompact AI is the microprocessor for AI".

When I was your age, the computer I worked on most was an IBM 1620. It had 10,000 decimal characters of memory. And you critically were dependent on punch cards because you had to run most things through and do intermediate storage and cache onto punch cards. And then we went on to the mainframe age. And I think what we're talking about today is equivalent for AI with the invention of the microprocessor. We lived in a mainframe age and that is where AI is today. Mainframe doesn't even begin to look at what Stargate is trying to do to go build its next generation model for Open AI. But it was a world, I actually used to teach a course on the social implications of the world when having it dominated so much by IBM and then later Microsoft. And as I agree with Dr. Diffie, decentralization is the key to freedom. Decentralization is also the key to getting the value out of the technology. I watch technology now. I started writing software 65 years ago. And so I've watched this over, you know, almost seven decades. And technology always finds its own level. But it's the ingenuity of people figuring out how to apply it to their day-to-day life problems that is what's going to make it valuable and where it goes. Yes, there are going to be some great things that are done, discovering new drugs, perhaps, that AI will enable that we haven't been able to do before. But what's key and what really changed the computer revolution was two things. One was the invention of the Internet. I actually built the first global IP network at SUN. and then the other things is distribution of computing. Today, I'm going to talk about the invention of the Internet. And as I said earlier, I think what's happening here today is the equivalent of the invention of the microprocessor. And it is the microprocessor which in the end enables some microsystems. Without the microprocessor, we never could have done what we did there. And then as those microprocessors got more powerful, we did more and more and more and more. And AI is going to do the same thing. Am I afraid of it? Yes. I mean, technology, you always have a two-edged sword. It always does things that are good, but people can also use it to do bad. And it has impacts on our society, which is the reason I wrote a book on that. But you're going to get the most value and assure the greatest value if we can distribute it and put it out there. And that is what microprocessor did. Everything we did today and the smart phones people carry all over the world. I mean, we have services with billions of users all talking to microprocessors. And it's going to be the same thing. And until we get the ability to use AI on CPUs and get it out so we can use the installed base, we're going to be hindered and we're going to have back to a world of IBM and Amdahl and a handful of other companies in a mainframe world. That's ending today. I think what you are seeing is this ability to broadly distribute inference and things so that we can have an use AI and everywhere in our lives and everywhere in our economies, everywhere in our societies. So I've been thrilled, I've been working with Ziroh Labs now for three years, and I think what they've done here and bring to market is profound, and we're going to see its impact for years ahead. Thank you very much.

"I think this discovery that AI can be done using ubiquitous CPUs rather than the rarer and inherently more expensive GPUs, It is the best, one of the most exciting things I have seen in quite a number of years".

The focus of my work, my desires throughout my entire career has been individual freedom. People often talk in terms of democracy, I can't see how you can have a free society without it being a democratic society, but it's amply evident that you can have a democratic society that is not a free society. And a lot of this has to do with the distribution of capability, productive power, decision-making, et cetera. And so my large interest is in decentralisation. Now, looking very broadly, look to biology, I'm very encouraged because the basic manufacturing process in biology is protein synthesis and occurs in every individual cell. On the other hand, if I look at the universe as a whole, I'm not so encouraged because the manufacture of heavy elements takes expensive objects Like supernovae and colliding neutron stars and it's very centralized. But looking to human constructs, the same issues apply if not the same grandiosity. We had as great a tool maybe as has ever occurred in human history, born approximately 35 years ago with the World Wide Web, the single biggest application running on the internet. And I believe that artificial intelligence, generative artificial intelligence, is the most powerful tool to have been developed since then. Now, I want to quote again from that UN report that was cited a few minutes ago because it was quoted by the Center for the AI and Digital Policy. It said, the UN warns of growing AI inequality as the market approaches 4.8 trillion With just 100 companies controlling 40% of R&D investment. The White House has issued federal AI guidance requiring chief AI officers, et cetera. And it's that first point, 100 companies controlling nearly half of the investment. And I think we are looking at a danger of having a small number of companies, and very frequently these things contract by mergers and goings out of business and so forth, might be many fewer than that, controlling artificial intelligence. Which means, roughly speaking, they will have to control the policy over what artificial intelligence is willing to do for customers. It would be much better If artificial intelligence is widely distributed, is accessible in all of its manifestations to many, many millions of people, and so I think this discovery that AI can be done using ubiquitous CPUs rather than the rarer and inherently more expensive GPUs, It is the best, one of the most exciting things I have seen in quite a number of years. So I salute this work and I hope to see it bloom.”

"So what we have today through Kompact AI is affordable, available, sustainable AI tools and techs on sustainability. I think thanks to Kompact AI, India is kind of getting its chance to innovate again".

I feel really privileged to be here today with you all. You know, it's interesting that this is a historic moment. We are making history. Let me repeat, today we are making history. And I am even more privileged that we are doing it from a very special institution, a marquee institution, IIT Madras. Hats off to IIT Madras, hats off to the Director IIT Madras, and hats off to Professor Kamakoti. I am separately mentioning Director IIT Madras and Professor Kamakoti because Professor Kamakoti is contributing in both capacities, both as a Director of the Institute as well as in his individual capacity. You already have seen what he covered and you will get to see as the day unfolds. So it was Professor C.K. Prahlad, the well-known management guru of University of Michigan, who in the year 2000 showcased to the world a series of frugal innovation, frugal innovations from India: Bombay Dabbawala in the West, Jaipur Foot in the North, Aravind Eye Hospital in the South, and a whole bunch of them. And each and every one of those frugal innovations, India demonstrated world-class solutions at very affordable costs, bringing the affordability to the center stage. And what we are doing today with the launch of Kompact AI, India is innovating again. India is innovating again, not in any field, but in a very special field in what is called the cutting edge, the bleeding edge, the deep tech AI. From today, AI development and AI deployment, let me repeat, AI development and AI deployment will be democratized in line with India's AI mission, AI for all. You know, tools from OpenAI, Google, Microsoft, AWS are critically depending on GPUs, as Professor Kamakoti mentioned. With limited availability and affordability of GPUs, on the top of it, sanctions from US, Indian researchers and professionals were at a disadvantage until today. India will also develop a GPU, but it will take some time. With the launch today by the Ziroh Labs of Kompact AI, AI solutions can be developed at a fraction of a cost using CPUs that are available in plenty, without the forced need of GPU. I'm not saying we will not use GPU, GPUs will be used, but we should not be forced to use GPUs. AI models today have the excessive use of electrical power using GPU, a point which Professor Kamakoti mentioned. And I think starting talking at the IIT Madras platform, we should kind of realize that the Indian wisdom has been talking of Dharma. You all know Dharma is at the root of civilizational living. There's a Sanskrit expression which says, Dharma dharigati prajah. Dharma is the one which sustains the civilization. And so it is the sustainability at the core when you actually have solutions that does not make too much dependence on electrical power, you are also providing sustainable solutions. So what we have today through Kompact AI is affordable, available, sustainable AI tools and techs on sustainability. I think thanks to Kompact AI, India is kind of getting its chance to innovate again. I feel particularly happy and proud that I have been kind of watching at close quarters, Ziroh Labs and Hrishikesh, the creators of Kompact AI from day minus one. It's a great privilege to be associated with them. A great day ahead and enjoy the rest of the day. Take a look at the demos. As they say, some of you should fall off your seats.

“The high cost of computing power, cloud services and proprietary tools is inhibiting the democratisation of AI today. But we know and believe with Kompact AI, this will change".

It's really, a privilege for me to be part of this, ceremony, you know, to talk about the rise of cost-effective AI unlocking exponential Opportunities. The key thing that I'd like to share with you, which I believe someone also mentioned, is that we stand at a pivotal moment. For years, AI's potential has been talked about but often accompanied, by concerns about high cost and complex implementation. But now the landscape is shifting, especially with the introduction of Kompact AI. We're entering an era where AI will become increasingly cost-effective, and this evolution is unlocking truly exponential opportunities. So let's talk about some relevant issues here. First, efficiency and optimization will be driven across different industries. Cost-effective AI has the ability to revolutionize industries. We know in manufacturing, it can optimize supply chains, and it can enable predictive maintenance, drastically reducing waste and downtime. Healthcare is another example where we have cost-effective AI accelerating things like drug discovery and personalized treatment that will lead to better outcomes at lower cost. In finance, I was at HSBC for well over 16 years. When we talk about detecting fraud and automating customer service and improving efficiency, that's another example where we can see the use of cost-effective AI. Secondly, another point, we're seeing democratization of AI. The high cost of computing power, cloud services and proprietary tools are inhibiting the democratization of AI today. Advanced AI development, as we see it, has remained largely with the top and the, the big tech giants. And this has prevented startups and small enterprises from fully leveraging its potential. This limits innovation and slows the development of diverse application. But we know with Kompact AI, this can change and will change. Another third point is bottlenecks to wider adoption that we see. AI models, as we all know, and as shared by several speakers today, rely heavily on GPUs. They're powerful, but they're expensive resources, which not only costly, but they're also limited in availability. So this makes scaling AI solutions a significant challenge. When we run these advanced models on CPUs, they will lead to performance bottlenecks, making real time or large-scale deployment nearly impossible. Techniques like, you know, that we know about, like quantization and distillation help reduce the computational load. But what usually happens is accuracy often takes a hit, making these solutions far from ideal. This poses a major problem. AI remains out of reach of many researchers, developers, and institutions. And this goes to the digital divide, that was talked about earlier on, especially those without access to elite infrastructure. But the increase in cost-effectiveness will help address all these problems, and that's where Kompact AI comes in. We believe that Kompact AI can help solve these problems. Kompact AI for us, and, you know, spending time with Hrishikesh and his team, is the first infrastructure product in the global market to unlock true value of AI on CPUs. Already we know the platform supports 17 foundational models, with plans to support over 100,later on this year. AI models will be available across thousands of data centers worldwide as a result of developers no longer having to wait for limited GPU resources. With this comparable speed of inference as expensive GPU chips, we believe Kompact AI's platform makes development for next generation AI applications faster, it will be cheaper, and highly accessible. This, will definitely foster a wave of innovation, driving new applications we haven't even imagined yet. So to bring it to a conclusion, breaking open cost-effective AI is not a future possibility, it's our present reality that must happen. And it's a welcome development to see IIT Madras and Kompact AI making this a reality. We now have a unique opportunity to shape the future, to drive the responsible and widespread adoption of AI that will benefit all some of these industries I mentioned and much more. But not just the industries, but society as a whole. So I encourage all of us, let us embrace this challenge and unlock the exponential opportunities that lie ahead. Thank you so much for letting me share these words with you.