Can Big Data and AI Turn Insights into Understanding?

By Ernest Worthman, IEEE Senior Member

December 5, 2017

Google posed this question recently, and it got me thinking about the intersection of big data and artificial intelligence. I have written a lot about both. Artificial intelligence has so many possibilities. In fact, I cannot think of any technological challenges where AI is not of benefit. In addition, there is little confusion or misunderstanding about what AI can do and where. However, big data is a horse of a different color, so to speak. While big data itself is fairly well understood regarding what it can do, unlike AI, it is somewhat ambiguous as to how to apply it and where.

Both AI and big data are on a fast evolutionary path. By themselves, they are impressive, but there has been little cross-pollination between the technologies thus far. That is a rather disappointing condition since the two technologies are so mutually complementary.

But it looks like that is changing and Google (of course) is at the head of the pack. They are looking to marry up AI and big data to gain some real meaningful perspective on a number of things.

There are several challenges with big data. I have discussed those in past missives and articles but essentially the issue is getting meaningful data from piles of raw data. There are lots of theories, but few working models. In addition, those that are working are a hybrid of current analysis apps with a few new approaches. However, not much in algorithmically-native big data platforms, as of yet.

The traditional algorithms that data scientists use do not port or scale well to big data so the development path has followed the creation of new algorithms and how to apply them to new data sets. That is where AI comes in.

Therefore, Google has started looking at this and they hit the nail right on the head with its idea of trying to understand big data by applying AI. Not that this is a new idea. It is advancements in AI that can apply these new and complex algorithms and spit the analysis out with much better metrics for further analysis. This may just be the missing link between acquiring and analyzing big data and being able to digest it.

The truth is that big data has been over hyped. Not what it can do, but when and how. It just has not been able to deliver what it is capable of, and for the reasons above. The application of AI may be the springboard that will get big data to deliver what it is promising.

Google seems to be sensing that. Because of the huge volumes, big data requires fast processing. That, in itself is not difficult. What is difficult is making some sense of what has been “fast processed.” Google seems to think AI is the solution to that.

To wit, self-driving cars. This platform is the epitome of the need for lots of data, processed fast with accurate results. Where AI comes into play here is its ability to cut through the clutter and recognize what is important – that is big – especially here, when one thinks about it. AI can do in moments what humans can do in hours, days, even weeks – and much more accurately. That is a must in self-driving cars. In the end, Google wants AI to remove much of the complexity of data analysis and bring it down to a human-manageable level. Now humans can address the real issues and leave the minutia to the machines.

Google just may have something here.

Ernest Worthman is the Executive Editor/Applied Wireless Technology. His 20-plus years of editorial experience includes being the Editorial Director of Wireless Design and Development and Fiber Optic Technology, the Editor of RF Design, the Technical Editor of Communications Magazine, Cellular Business, Global Communications and a Contributing Technical Editor to Mobile Radio Technology, Satellite Communications, as well as computer-related periodicals such as Windows NT. His technical writing practice client list includes RF Industries, GLOBALFOUNDRIES, Agilent Technologies, Advanced Linear Devices, Ceitec, SA, and others. Before becoming exclusive to publishing, he was a computer consultant and regularly taught courses and seminars in applications software, hardware technology, operating systems, and electronics. Ernest’s client list has included Lucent Technologies, Jones Intercable, Qwest, City and County of Denver, TCI, Sandia National Labs, Goldman Sachs, and other businesses. His credentials include a BS, Electronic Engineering Technology; A.A.S, Electronic Digital Technology. He has held a Colorado Post-Secondary/Adult teaching credential, member of IBM’s Software Developers Assistance Program and Independent Vendor League, a Microsoft Solutions Provider Partner, and a life member of the IEEE. He has been certified as an IBM Certified OS2 consultant and trainer; WordPerfect Corporation Developer/Consultant and Lotus Development Corporation Developer/Consultant. He was also a first-class FCC technician in the early days of radio. Ernest Worthman may be contacted at: eworthman@aglmediagroup.com.