Posts

As Businesses Struggle With ML, Automation Offers a Solution

In recent years, machine learning technology and the business solutions it enables has developed into a big business in and of itself. According to the industry analysts at IDC, spending on ML and AI technology is set to grow to almost $98 billion per year by 2023. In practical terms, that figure represents a business environment where ML technology has become a key priority for companies of every kind.

That doesn’t mean that the path to adopting ML technology is easy for businesses. Far from it. In fact, survey data seems to indicate that businesses are still struggling to get their machine learning efforts up and running. According to one such survey, it currently takes the average business as many as 90 days to deploy a single machine learning model. For 20% of businesses, that number is even higher.

From the data, it seems clear that something is missing in the methodologies that most companies rely on to make meaningful use of machine learning in their business workflows. A closer look at the situation reveals that the vast majority of data workers (analysts, data scientists, etc.) spend an inordinate amount of time on infrastructure work – and not on creating and refining machine learning models.

Streamlining the ML Adoption Process

To fix that problem, businesses need to turn to another growing area of technology: automation. By leveraging the latest in automation technology, it’s now possible to build an automated machine learning pipeline (AutoML pipeline) that cuts down on the repetitive tasks that slow down ML deployments and lets data workers get back to the work they were hired to do. With the right customized solution in place, a business’s ML team can:

  • Reduce the time spent on data collection, cleaning, and ingestion
  • Minimize human errors in the development of ML models
  • Decentralize the ML development process to create an ML-as-a-service model with increased accessibility for all business stakeholders

In short, an AutoML pipeline turns the high-effort functions of the ML development process into quick, self-adjusting steps handled exclusively by machines. In some use cases, an AutoML pipeline can even allow non-technical stakeholders to self-create ML solutions tailored to specific business use cases with no expert help required. In that way, it can cut ML costs, shorten deployment time, and allow data scientists to focus on tackling more complex modelling work to develop custom ML solutions that are still outside the scope of available automation techniques.

The Parts of an AutoML Pipeline

Although the frameworks and tools used to create an AutoML pipeline can vary, they all contain elements that conform to the following areas:

  • Data Preprocessing – Taking available business data from a variety of sources, cleaning it, standardizing it, and conducting missing value imputation
  • Feature Engineering – Identifying features in the raw data set to create hypotheses for the model to base predictions on
  • Model Selection – Choosing the right ML approach or hyperparameters to produce the desired predictions
  • Tuning Hyperparameters – Determining which hyperparameters help the model achieve optimal performance

As anyone familiar with ML development can tell you, the steps in the above process tend to represent the majority of the labour and time-intensive work that goes into creating a model that’s ready for real-world business use. It is also in those steps where the lion’s share of business ML budgets get consumed, and where most of the typical delays occur.

The Limitations and Considerations for Using AutoML

Given the scope of the work that can now become part of an AutoML pipeline, it’s tempting to imagine it as a panacea – something that will allow a business to reduce its reliance on data scientists going forward. Right now, though, the technology can’t do that. At this stage, AutoML technology is still best used as a tool to augment the productivity of business data teams, not to supplant them altogether.

To that end, there are some considerations that businesses using AutoML will need to keep in mind to make sure they get reliable, repeatable, and value-generating results, including:

  • Transparency – Businesses must establish proper vetting procedures to make sure they understand the models created by their AutoML pipeline, so they can explain why it’s making the choices or predictions it’s making. In some industries, such as in medicine or finance, this could even fall under relevant regulatory requirements.
  • Extensibility – Making sure the AutoML framework may be expanded and modified to suit changing business needs or to tackle new challenges as they arise.
  • Monitoring and Maintenance – Since today’s AutoML technology isn’t a set-it-and-forget-it proposition, it’s important to establish processes for the monitoring and maintenance of the deployment so it can continue to produce useful and reliable ML models.

The Bottom Line

As it stands today, the convergence of automation and machine learning holds the promise of delivering ML models at scale for businesses, which would greatly speed up the adoption of the technology and lower barriers to entry for those who have yet to embrace it. On the whole, that’s great news both for the businesses that will benefit from increased access to ML technology, as well as for the legions of data professionals tasked with making it all work.

It’s important to note, of course, that complete end-to-end ML automation with no human intervention is still a long way off. While businesses should absolutely explore building an automated machine learning pipeline to speed up development time in their data operations, they shouldn’t lose sight of the fact that they still need plenty of high-skilled data scientists and analysts on their teams. It’s those specialists that can make appropriate and productive use of the technology. Without them, an AutoML pipeline would accomplish little more than telling the business what it wants to hear.

The good news is that the AutoML tools that exist right now are sufficient to alleviate many of the real-world problems businesses face in their road to ML adoption. As they become more commonplace, there’s little doubt that the lead time to deploy machine learning models is going to shrink correspondingly – and that businesses will enjoy higher ROI and enhanced outcomes as a result.

Interview – There is no stand-alone strategy for AI, it must be part of the company-wide strategy

Ronny FehlingRonny Fehling is Partner and Associate Director for Artificial Intelligence as the Boston Consulting Group GAMMA. With more than 20 years of continually progressive experience in leading business and technology innovation, spearheading digital transformation, and aligning the corporate strategy with Artificial Intelligence he industry-leading organizations to grow their top-line and kick-start their digital transformation.

Ronny Fehling is furthermore speaker of the Predictive Analytics World for Industry 4.0 in May 2020.

Data Science Blog: Mr. Fehling, you are consulting companies and business leaders about AI and how to get started with it. AI as a definition is often misleading. How do you define AI?

This is a good question. I think there are two ways to answer this:

From a technical definition, I often see expressions about “simulation of human intelligence” and “acting like a human”. I find using these terms more often misleading rather than helpful. I studied AI back when it wasn’t yet “cool” and still middle of the AI winter. And yes, we have much more compute power and access to data, but we also think about data in a very different way. For me, I typically distinguish between machine learning, which uses algorithms and statistical methods to identify patterns in data, and AI, which for me attempts to interpret the data in a given context. So machine learning can help me identify and analyze frequency patterns in text and even predict the next word I will type based on my history. AI will help me identify ‘what’ I’m writing about – even if I don’t explicitly name it. It can tell me that when I’m asking “I’m looking for a place to stay” that I might want to see a list of hotels around me. In other words: machine learning can detect correlations and similar patterns, AI uses machine learning to generate insights.

I always wondered why top executives are so frequently asking about the definition of AI because at first it seemed to me not as relevant to the discussion on how to align AI with their corporate strategy. However, I started to realize that their question is ultimately about “What is AI and what can it do for me?”.

For me, AI can do three things really good, which humans cannot really do and previous approaches couldn’t cope with:

  1. Finding similar patterns in historical data. Imagine 20 years of data like maintenance or repair documents of a manufacturing plant. Although they describe work done on a multitude of products due to a multitude of possible problems, AI can use this to look for a very similar situation based on a current problem description. This can be used to identify a common root cause as well as a common solution approach, saving valuable time for the operation.
  2. Finding correlations across time or processes. This is often used in predictive maintenance use cases. Here, the AI tries to see what similar events happen typically at some time before a failure happen. This way, it can alert the operator much earlier about an impending failure, say due to a change in the vibration pattern of the machine.
  3. Finding an optimal solution path based on many constraints. There are many problems in the business world, where choosing the optimal path based on complex situations is critical. Let’s say that suddenly a severe weather warning at an airport forces an airline to have to change their scheduling because of a reduced airport capacity. Delays for some aircraft can cause disruptions because passengers or personnel not being able to connect anymore. Knowing which aircraft to delay, which to cancel, which to switch while causing the minimal amount of disruption to passengers, crew, maintenance and ground-crew is something AI can help with.

The key now is to link these fundamental capabilities with the business context of the company and how it can ultimately help transform.

Data Science Blog: Companies are still starting with their own company-wide data strategy. And now they are talking about AI strategies. Is that something which should be handled separately?

In my experience – both based on having seen the implementations of several corporate data strategies as well as my upbringing at Oracle – the data strategy and AI strategy are co-dependent and cannot be separated. Very often I hear from clients that they think they first need to bring their data in order before doing AI project. And yes, without good data access, AI cannot really work. In fact, most of the time spent on AI is spent on processing, cleansing, understanding and contextualizing the data. However, you cannot really know what data will be needed in which form without knowing what you want to use it for. This is why strategies that handle data and AI separately mostly fail and generate huge costs.

Data Science Blog: What are the important steps for developing a good data strategy? Is there something like a general approach?

In my eyes, the AI strategy defines the data strategy step by step as more use cases are implemented. Rather than focusing too quickly at how to get all corporate data into a data lake, it will be much more important to start creating a use-case, technology and data governance. This governance has to be established once the AI strategy is starting to mature to enable the scale up and productization. At the beginning is to find the (very few) use-cases that can serve as light house projects to demonstrate (1) value impact, (2) a way to go from MVP to Pilot, and (3) how to address the data challenge. This will then more naturally identify the elements of governance, data access and technology that are required.

Data Science Blog: What are the most common questions from business leaders to you regarding AI? Why do they hesitate to get started?

By far it the most common question I get is: how do I get started? The hesitations often come from multiple sources like: “We don’t have the talent in house to do AI”, “Our data is not good enough”, “We don’t know which use-case to start with”, “It’s not easy for us to embrace agile and failure culture because our products are mission critical”, “We don’t know how much value this can bring us”.

Data Science Blog: Most managers prefer to start small and with lower risk. They seem to postpone bigger ideas to a later stage, at least some milestones should be reached. Is that a good idea or should they think bigger?

AI is often associated (rightfully so) with a new way of working – agile and embracing failures. Similarly, there is also the perception of significant cost to starting with AI (talent, technology, data). These perceptions often lead managers wanting to start with several smaller ambition use-cases where failure isn’t that grave. Once they have proven itself somehow, they would then move on to bigger projects. The problem with this strategy is on the one side that you fragment your few precious AI resources on too many projects and at the same time you cannot really demonstrate an impact since the projects weren’t chosen based on their impact potential.

The AI pioneers typically were successful by “thinking big, starting small and scaling fast”. You start by assessing the value potential of a use-case, for example: my current OEE (Overall Equipment Efficiency) is at 65%. There is an addressable loss of 25% which would grow my top line by $X. With the help of AI experts, you then create a hypothesis of how you think you can reduce that loss. This might be by choosing one specific equipment and 50% of the addressable loss. This is now the measure against which you define your failure or non-failure criteria. Once you have proven an MVP that can solve this loss, you scale up by piloting it in real-life setting and then scaling it to all the equipment. At every step of this process, you have a failure criterion that is measured by the impact value.


Virtual Edition, 11-12 MAY, 2020

The premier machine learning
conference for industry 4.0

This year Predictive Analytics World for Industry 4.0 runs alongside Deep Learning World and Predictive Analytics World for Healthcare.

Simple RNN

A gentle introduction to the tiresome part of understanding RNN

Just as a normal conversation in a random pub or bar in Berlin, people often ask me “Which language do you use?” I always answer “LaTeX and PowerPoint.”

I have been doing an internship at DATANOMIQ and trying to make straightforward but precise study materials on deep learning. I myself started learning machine learning in April of 2019, and I have been self-studying during this one-year-vacation of mine in Berlin.

Many study materials give good explanations on densely connected layers or convolutional neural networks (CNNs). But when it comes to back propagation of CNN and recurrent neural networks (RNNs), I think there’s much room for improvement to make the topic understandable to learners.

Many study materials avoid the points I want to understand, and that was as frustrating to me as listening to answers to questions in the Japanese Diet, or listening to speeches from the current Japanese minister of the environment. With the slightest common sense, you would always get the feeling “How?” after reading an RNN chapter in any book.

This blog series focuses on the introductory level of recurrent neural networks. By “introductory”, I mean prerequisites for a better and more mathematical understanding of RNN algorithms.

I am going to keep these posts as visual as possible, avoiding equations, but I am also going to attach some links to check more precise mathematical explanations.

This blog series is composed of five contents.:

  1. Why Do We Need RNN?(to be published soon)
  2. Simple Structure of RNN and Its Forward Propagation (to be published soon)
  3. Back Propagation of RNN (to be published soon)
  4. Vanishing Gradient Problem and Winter of AI Research (to be published soon)
  5. LSTM(Long Short-Term Memory) and Its Back Propagation (to be published soon)

 

Krisenerkennung und -bewältigung mit Daten und KI

Wie COVID-19 unser Verständnis für Daten und KI verändert

Personenbezogene Daten und darauf angewendete KI galten hierzulande als ein ganz großes Pfui. Die Virus-Krise ändert das – Zurecht und mit großem Potenzial auch für die Wirtschaft.

Aber vorab, wie hängen Daten und Künstliche Intelligenz (KI) eigentlich zusammen? Dies lässt sich einfach und bildlich erläutern, denn Daten sind sowas wie der Rohstoff für die KI als Motor. Und dieser Motor ist nicht nur als Metapher zu verstehen, denn KI bewegt tatsächlich etwas, z. B. automatisierte Prozesse in Marketing, Vertrieb, Fertigung, Logistik und Qualitätssicherung. KI schützt vor Betrugsszenarien im Finanzwesen oder Ausfallszenarien in der produzierenden Industrie.

KI schützt jeden Einzelnen aber auch vor fehlenden oder falschen Diagnosen in der Medizin und unsere Gesellschaft vor ganzen Pandemien. Das mag gerade im Falle des SARS-COV-2 in 2019 in der VR China und 2020 in der ganzen Welt noch nicht wirklich geklappt zu haben, aber es ist der Auslöser und die Probe für die nun vermehrten und vor allem den verstärkten Einsatz von KI als Spezial- und Allgemein-Mediziner.

KI stellt spezielle Diagnosen bereits besser als menschliche Gehirne es tun

Menschliche Gehirne sind wahre Allrounder, sie können nicht nur Mathematik verstehen und Sprachen entwickeln und anwenden, sondern auch Emotionen lesen und vielfältige kreative Leistungen vollbringen. Künstliche Gehirne bestehen aus programmierbaren Schaltkreisen, die wir über mehrere Abstraktionen mit Software steuern und unter Einsatz von mathematischen Methoden aus dem maschinellen Lernen gewissermaßen auf die Mustererkennung abrichten können. Diese gerichteten Intelligenzen können sehr viel komplexere Muster in sehr viel mehr und heterogenen Daten erkennen, die für den Menschen nicht zugänglich wären. Diesen Vorteil der gerichteten künstlichen Intelligenz werden wir Menschen nutzen – und tun es teilweise schon heute – um COVID-19 automatisiert und sehr viel genauer anhand von Röntgen-Bildern zu erkennen.

Dies funktioniert in speziellen Einsätzen auch für die Erkennung von verschiedenen anderen Lungen-Erkrankungen sowie von Knochenbrüchen und anderen Verletzungen sowie natürlich von Krebs und Geschwüren.

Die Voraussetzung dafür, dass dieser Motor der automatisierten und akkuraten Erkennung funktioniert, ist die Freigabe von vielen Daten, damit die KI das Muster zur Diagnose erlernen kann.

KI wird Pandemien vorhersagen

Die Politik in Europa steht viel in der Kritik, möglicherweise nicht richtig und rechtzeitig auf die Pandemie reagiert zu haben. Ein Grund dafür mögen politische Grundprinzipien sein, ein anderer ist sicherlich das verlässliche Vorhersage- und Empfehlungssystem für drohende Pandemien. Big Data ist der Treibstoff, der diese Vorhersage-Systeme mit Mustern versorgt, die durch Verfahren des Deep Learnings erkannt und systematisch zur Generalisierung erlernt werden können.

Um viele Menschenleben und darüber hinaus auch berufliche Existenzen zu retten, darf der Datenschutz schon mal Abstriche machen. So werden beispielsweise anonymisierte Standort-Daten von persönlichen Mobilgeräten an das Robert-Koch-Institut übermittelt, um die Corona-Pandemie besser eindämmen zu können. Hier haben wir es tatsächlich mit Big Data zutun und die KI-Systeme werden besser, kämen auch noch weitere Daten zur medizinischen Versorgung, Diagnosen oder Verkehrsdaten hinzu. Die Pandemie wäre transparenter als je zuvor und Virologen wie Alexander Kekulé von der Martin-Luther-Universität in Halle-Wittenberg haben die mathematische Vorhersagbarkeit schon häufig thematisiert. Es fehlten Daten und die Musterkennung durch die maschinellen Lernverfahren, die heute dank aktiver Forschung in Software und Hardware (Speicher- und Rechenkapazität) produktiv eingesetzt werden können.

Übrigens darf auch hier nicht zu kurz gedacht werden: Auch ganz andere Krisen werden früher oder später Realität werden, beispielsweise Energiekrisen. Was früher die Öl-Krise war, könnten zukünftig Zusammenbrüche der Stromnetze sein. Es braucht nicht viel Fantasie, dass KI auch hier helfen wird, Krisen frühzeitig zu erkennen, zu verhindern oder zumindest abzumildern.

KI macht unseren privaten und beruflichen Alltag komfortabler und sicherer

Auch an anderer Front kämpfen wir mit künstlicher Intelligenz gegen Pandemien sozusagen als Nebeneffekt: Die Automatisierung von Prozessen ist eine Kombination der Digitalisierung und der Nutzung der durch die digitalen Produkte genierten Daten. So werden autonome Drohnen oder autonome Fahrzeuge vor allem im Krisenfall wichtige Lieferungen übernehmen und auch Bezahlsysteme bedingen keinen nahen menschlichen Kontakt mehr. Und auch Unternehmen werden weniger Personal physisch vor Ort am Arbeitsplatz benötigen, nicht nur dank besserer Telekommunikationssysteme, sondern auch, weil Dokumente nur noch digital vorliegen und operative Prozesse datenbasiert entschieden und dadurch automatisiert ablaufen.

So blüht uns also eine schöne neue Welt ohne Menschen? Nein, denn diese werden ihre Zeit für andere Dinge und Berufe einsetzen. Menschen werden weniger zur roboter-haften Arbeitskraft am Fließband, an der Kasse oder vor dem Steuer eines Fahrzeuges, sondern sie werden menschlicher, denn sie werden sich entweder mehr mit Technologie befassen oder sich noch sozialere Tätigkeiten erlauben können. Im Krisenfall jedoch, werden wir die dann unangenehmeren Tätigkeiten vor allem der KI überlassen.

AI For Advertisers: How Data Analytics Can Change The Maths Of Advertising?

All Images Credit: Freepik

The task of understanding a customer’s journey and designing your marketing strategy accordingly can be difficult in this data-driven world. Today, the customer expresses their needs in myriad forms of requests.

Consumers express their needs and want attitudes, and values in various forms through search, comments, blogs, Tweets, “likes,” videos, and conversations and access such data across many channels like web, mobile, and face to face. Volume, variety, velocity and veracity of the data accumulated through these customer interactions are huge.

BigData and data analytics can be leveraged to understand several phases of the customer journey. There are risks involved in using Artificial Intelligence for the marketing data analysis of data breach and even manipulation. But, AI do have brighter prospects when it comes to marketing and advertiser applications.

As the CEO of a technology firm Chop Dawg and marketer, Joshua Davidson puts it, “AI-powered apps are going to be the future for us, and there are several industries that are ripe for this.” The mobile-first strategy of many enterprises has powered the use of AI for digital marketing and developing technologies and innovations to power industries with intelligent systems.

How AI and Machine learning are affecting customer journeys?

Any consumer journey begins with the recognition of a problem and then stages like initial consideration, active evaluation, purchase, and postpurchase come through up till the consumer journey is over. The need for identifying the purchasing and need patterns of the consumers and finding the buyer personas to strategize the marketing for them.

Need and Want Recognition:

Identifying a need is quite difficult as it is the most initial level of a consumer’s journey and it is more on the category level than at a brand level. Marketers and advertisers are relying on techniques like market research, web analytics, and data mining to build consumer profiles and buyer’s persona for understanding the needs and influencing the purchase of products. AI can help identify these wants and needs in real-time as the consumers usually express their needs and wants online and help build profiles more quickly.

AI technologies offered by several firms help in consumer profiling. Firms like Microsoft offers Azure that crunches billions of data points in seconds to determine the needs of consumers. It then personalizes web content on specific platforms in real-time to align with those status-updates. Consumer digital footprints are evolving through social media status updates, purchasing behavior, online comments and posts. Ai tends to update these profiles continuously through machine learning techniques.

Initial Consideration:

A key objective of advertising is to insert a brand into the consideration set of the consumers when they are looking for deliberate offerings. Advertising includes increasing the visibility of brands and emphasize on the key reasons for consideration. Advertisers currently use search optimization, paid search advertisements, organic search, or advertisement retargeting for finding the consideration and increase the probability of consumer consideration.

AI can leverage machine learning and data analytics to help with search, identify and rank functions of consumer consideration that can match the real-time considerations at any specific time. Take an example of Google Adwords, it analyzes the consumer data and helps advertisers make clearer distinctions between qualified and unqualified leads for better targeting.

Google uses AI to analyze the search-query data by considering, not only the keywords but also context words and phrases, consumer activity data and other BigData. Then, Google identifies valuable subsets of consumers and more accurate targeting.

Active Evaluation: 

When consumers narrow it down to a few choices of brands, advertisers need to insert trust and value among the consumers for brands. A common technique is to identify the higher purchase consumers and persuade them through persuasive content and advertisement. AI can support these tasks using some techniques:

Predictive Lead Scoring: Predictive lead scoring by leveraging machine learning techniques of predictive analytics to allow marketers to make accurate predictions related to the intent of purchase for consumers. A machine learning algorithm runs through a database of existing consumer data, then recognize trends and patterns and after processing the external data on consumer activities and interests, creates robust consumer profiles for advertisers.

Natural Language Generation: By leveraging the image, speech recognition and natural language generation, machine learning enables marketers to curate content while learning from the consumer behavior in real-time scenarios and adjusts the content according to the profiles on the fly.

Emotion AI: Marketers use emotion AI to understand consumer sentiment and feel about the brand in general. By tapping into the reviews, blogs or videos they understand the mood of customers. Marketers also use emotion AI to pretest advertisements before its release. The famous example of Kelloggs, which used emotion AI to help devise an advertising campaign for their cereal, eliminating the advertisement executions whenever the consumer engagement dropped.

Purchase: 

As the consumers decide which brands to choose and what it’s worth, advertising aims to move them out of the decision process and push for the purchase by reinforcing the value of the brand compared with its competition.

Advertisers can insert such value by emphasizing convenience and information about where to buy the product, how to buy the product and reassuring the value through warranties and guarantees. Many marketers also emphasize on rapid return policies and purchase incentives.

AI can completely change the purchase process through dynamic pricing, which encompasses real-time price adjustments on the basis of information such as demand and other consumer-behavior variables, seasonality, and competitor activities.

Post-Purchase: 

Aftersales services can be improved through intelligent systems using AI technologies and machine learning techniques. Marketers and advertisers can hire dedicated developers to design intelligent virtual agents or chatbots that can reinforce the value and performance of a brand among consumers.

Marketers can leverage an intelligent technique known as Propensity modeling to identify the most valuable customers on the basis of lifetime value, likelihood of reengagement, propensity to churn, and other key performance measures of interest. Then advertisers can personalize their communication with these customers on the basis of these data.

Conclusion:

AI has shifted the focus of advertisers and marketers towards the customer-first strategies and enhanced the heuristics of customer engagement. Machine learning and IoT(Internet of Things) has already changed the way customer interact with the brands and this transition has come at a time when advertisers and marketers are looking for new ways to tap into the customer mindset and buyer’s persona.

All Images Credit: Freepik

AI Experts: The Next Frontier in AI After the 2020 Job Crisis

Beware the perils of AI boom!

Isn’t this something that should ring alarm bells to upgrade your AI skills.

Artificial intelligence has grown smarter putting people in awe with a question, “Is my job safe?” Should we be afraid? It is but a simple question with a rather perplexing answer, I’m not skilled ready. Your view will depend on whether you’ll be able to develop skills that will surpass the redundant skills you possess today.No doubt, the AI domain is thriving and humans are scared. 

Even organizations such as McKinsey predicts the doom and gloom scenario where one-third of the workers’ jobs will be taken over due to automation by 2030.

In the next decade, AI and automation could banish 54 million Americans out of their workspace. With rapid technological growth, machines are now outperforming the number of tasks traditionally done by manpower.

What’s more?

  • Walmart has the fastest automated truck unloader that helps scan unloaded items on a priority basis. 
  • McDonald replaces drive-thru workers with order-taking AI and cashiers along with self-checkout kiosks. 
  • While farms in California hire robots to harvest lettuce. 

Fear facts appearing real

Near about 670,000 U.S. jobs were replaced between 1990 to 2007, mostly in the manufacturing sector. But this trend is already accelerating as it advances in mobile technology, data transfer, AI, and computing speed. 

On its face, jobs that involve physical tasks in predictable environments will be at higher risk. For instance, The Palm Beach County Court recently made use of four robots (Rosie Tobor, Kitt Robbie, Speedy, and Wally Bishop) to read out the court filings, input data into the case management system, and fill out docket sheets. Also, at certain places in China, waiters were being replaced with robots.

On the contrary, jobs that include creative thinking, social interaction, and managing people will barely involve automation.

Though you think your job is safe, it isn’t. 

History has warned us of the apocalyptic happenings about technology replacing our jobs. There has always been a difficult transition to jobs that require newer skillsets. McKinsey, in its study, mentioned 8-9% of 2030’s labor demand will be in newer job roles that did not exist before. 

AI to take over the world – or is it? 

There is still but a grim prognostic about the robot apocalypse. But it’s not the time to celebrate.

As warned by Russian president Vladimir Putin, “The nation that leads in AI will be the ruler of the world.”

Artificial intelligence is yet to replace the human workforce, but it is still considered an invaluable tool today.

According to Forrester, firms will now address the pragmatic side of AI about having a better understanding of the challenges faced, to embrace the idea which is, no pain means no AI gain. The AI reality is here, right now. Organizations have now realized what they can do and what they cannot. Their focus is now projected toward taking proactive measures to produce more AI talents like AI experts and AI specialists, etc. 

Is there a timeframe where AI will overtake the human race?

It is only a matter of time when artificial intelligence will become smarter than its human creators.

Experts have already started to build a world that is brimming with AI. But sadly, in the present, most individuals are yet to know what AI even is. By the next decade, AI is predicted to outperform human in multiple activities such as,

  • Translating languages – 2024
  • Writing high-school essays – 2025
  • Driving a truck – 2027
  • Working in the retail sector – 2031
  • Or writing a best-selling book – 2049
  • Work as a surgeon – 2053

Beyond the shadow of a doubt, as artificial intelligence continues to grow, some experts say we’ll eventually hit the plateau. On the research side, there will be a snowball of AI challenges. Therefore, to tackle these challenges, the demand for AI experts will dramatically upsurge.

In addition to the dearth of AI talent, the transition may bring new challenges both for policymakers as well as AI professionals. 

“High-level machine intelligence will start performing any task better than the humans by 2060, and will take away human jobs by 2136, predicted a study done by multiple researchers from Yale and Oxford University.”

To stay prepared for the upcoming challenges, upskilling is the right way to reshape and overcome the AI jobs crisis. 

Upskilling in AI is the new mandate

Notably, as AI takes on to become the next technology revolution, certifications in artificial intelligence will keep you one step ahead. 

The advent of artificial intelligence has advanced at a level where there is a dire need for AI engineers. Now is the right time to pursue a career in artificial intelligence.

The current job market is flooded with multiple AI career options, but there’s a significant dearth of talent in the AI field. Professionals like software engineers have an upper hand in the AI industry. Additional certification programs have the capability of boosting the credibility of such individuals. 

Just like any other technology predictions, it’s an ideal decision to take up AI certifications. Staying up-to-date will prevent you from unnecessary panic – where AI could help you and not hurt you.

An economist Yale Brozen from the University of Chicago found out about technology destroying approximately 12 million jobs in the 1950s. But consecutively it also created over 20 million jobs as vast productivity leading toward the demand for more workers to keep up the pace with the rising demand.

Do you still need a reason not to adopt AI?

The AI catastrophe that dooms us is a threat to humans today. The pronouncement has retreated into a grim future where ignorance is not the solution. 

The pervasive answer is, only individuals that can make progress in their AI career will make it through the job crisis. 

Do you think your job is safe? Think again!

Why Retailers Are Making the Push for Stronger Data Science and AI

Retail relies on what the customer wants and needs at that moment, no matter the size of the company. Making judgments without consumer input would probably work for a little while but will fall flat as soon as the business model becomes outdated. In today’s technology-run world, things can become obsolete in a matter of days or even hours.

Retailers are the businesses most in need of capitalizing on what the customer wants in real-time. They have started to use data science and information from the Internet of Things (IoT) to not only stay in business, but also get ahead of other brands.

Artificial intelligence (AI) adds a new layer by using modern technology. The details of why retailers want to use these new practices are a bit more specific, though.

Data Targets Audiences

By using current customer data compared to information from the IoT, retailers can learn more about their audience and find better means of targeting them. Demographics like age, location and many other factors could affect advertising and even shopping, not to mention holidays throughout the year an audience celebrates.

Websites also need to be customized to suit the target audience. Those that are mobile-friendly and focused on what shoppers want can increase revenue, but the wrong approach can drive away new and existing customers. AI can help companies understand that data and present it back to the customer seamlessly, providing different options for various audiences.

Customer Base Expansion

Customer success should mean business success, as well. Growing a client base is something data science can assist with. However, helping customers grow is another type of service few companies provide but all people appreciate. A business can expand by offering new products and services that are relevant to their audience through the use of data.

Once a company learns what current customers want and begin to fit their needs, it can expand to more audiences. With data science, a business can ensure it does so slowly to give more of what current customers want while also finding new ones. The data can tell what sort of interests they all share so companies can capitalize on the venture.

AI Helps Customer Service

AI helps out customer service on both ends. Employees don’t have to focus on common problems that could easily be resolved, and clients often walk away happier than if they were to speak to a real person. This doesn’t work for every problem, especially ones that are specific in nature, but they can assist with more common issues. This is where chatbots enter the stage.

An AI-supported chatbot can give immediate support, provide suggestions, answer direct questions and offer almost any other form of help needed. Customers get personalized attention, and businesses can work faster toward customer loyalty.

Again, speaking to a real person when they have problems is a big plus for customers, but not for issues they know could be resolved in the time it takes to wait on the line for a representative.

Supply and Demand

Price optimization has taken on a bigger role than it has in the past. Mostly, data science is looking at supply and demand in real-time rather than having price fluctuations occur months after the business loses money. Having the right price can also help create more promotions for products and services, rewarding loyal customers for their shopping.

The data has to be gained from multiple channels by using price optimization tools, which focus on using data correctly in a company’s favor. The information doesn’t just look at supply and demand, but also examines locations, times, customer attitudes, competitor pricing and many other factors. All these pieces of information can be delivered in real-time so prices can be changed accordingly.

Taking the Competition

The thing about data science is that businesses are already utilizing it to their full potential and getting more customers than ever. The only way to get ahead of the competition is to at least start using the tools they’ve had at their disposal for years.

Target was one such company that took up the data helm. During 2012 and 2013, it saw a pretty sizeable dip in sales, but its online sales went up by almost 30% during the same time.

Data and Retail

When running a retail business, especially one that’s branching off into a franchise, using data is imperative. Data science and AI have become extremely important to companies both big and small.

Applying it correctly can help enterprises of any size and in every industry take things to the next level.

Even if a company is just starting out, sticking the first landing with a target audience is a fantastic way to begin the adventure and find success.

Glorious career paths of a Big Data Professional

Are you wondering about the career profiles you may get to fill if you get into Big Data industry? If yes, then Bingo! This is the post that will inform you just about that. Big data is just an umbrella term. There are a lot of profiles and career paths that are covered under this umbrella term. Let us have a look at some of these profiles.

Data Visualisation Specialist

The process of visualizing data is turning out to be critical in guaranteeing information-driven representatives get the upfront investment required to actualize goal-oriented and significant Big Data extends in their organization. Making your data to tell a story and the craft of envisioning information convincingly has turned into a significant piece of the Big Data world and progressively associations need to have these capacities in-house. Besides, as a rule, these experts are relied upon to realize how to picture in different instruments, for example, Spotfire, D3, Carto, and Tableau – among numerous others. Information Visualization Specialists should be versatile and inquisitive to guarantee they stay aware of most recent patterns and answers for a recount to their information stories in the most intriguing manner conceivable with regards to the board room. 

 

Big Data Architect

This is the place the Hadoop specialists come in. Ordinarily, a Big Data planner tends to explicit information issues and necessities, having the option to portray the structure and conduct of a Big Data arrangement utilizing the innovation wherein they practice – which is, as a rule, mostly Hadoop.

These representatives go about as a significant connection between the association (and its specific needs) and Data Scientists and Engineers. Any organization that needs to assemble a Big Data condition will require a Big Data modeler who can serenely deal with the total lifecycle of a Hadoop arrangement – including necessity investigation, stage determination, specialized engineering structure, application plan, and advancement, testing the much-dreaded task of deploying lastly.

Systems Architect 

This Big data professional is in charge of how your enormous information frameworks are architected and interconnected. Their essential incentive to your group lies in their capacity to use their product building foundation and involvement with huge scale circulated handling frameworks to deal with your innovation decisions and execution forms. You’ll need this individual to construct an information design that lines up with the business, alongside abnormal state anticipating the improvement. The person in question will consider different limitations, adherence to gauges, and varying needs over the business.

Here are some responsibilities that they play:

    • Determine auxiliary prerequisites of databases by investigating customer tasks, applications, and programming; audit targets with customers and assess current frameworks.
    • Develop database arrangements by planning proposed framework; characterize physical database structure and utilitarian abilities, security, back-up and recuperation particulars.
    • Install database frameworks by creating flowcharts; apply ideal access methods, arrange establishment activities, and record activities.
    • Maintain database execution by distinguishing and settling generation and application advancement issues, figuring ideal qualities for parameters; assessing, incorporating, and putting in new discharges, finishing support and responding to client questions.
    • Provide database support by coding utilities, reacting to client questions, and settling issues.


Artificial Intelligence Developer

The certain promotion around Artificial Intelligence is additionally set to quicken the number of jobs publicized for masters who truly see how to apply AI, Machine Learning, and Deep Learning strategies in the business world. Selection representatives will request designers with broad learning of a wide exhibit of programming dialects which loan well to AI improvement, for example, Lisp, Prolog, C/C++, Java, and Python.

All said and done; many people estimate that this popular demand for AI specialists could cause a something like what we call a “Brain Drain” organizations poaching talented individuals away from the universe of the scholarly world. A month ago in the Financial Times, profound learning pioneer and specialist Yoshua Bengio, of the University of Montreal expressed: “The industry has been selecting a ton of ability — so now there’s a lack in the scholarly world, which is fine for those organizations. However, it’s not extraordinary for the scholarly world.” It ; howeverusiasm to perceive how this contention among the scholarly world and business is rotated in the following couple of years.

Data Scientist

The move of Big Data from tech publicity to business reality may have quickened, yet the move away from enrolling top Data Scientists isn’t set to change in 2020. An ongoing Deloitte report featured that the universe of business will require three million Data Scientists by 2021, so if their expectations are right, there’s a major ability hole in the market. This multidisciplinary profile requires specialized logical aptitudes, specialized software engineering abilities just as solid gentler abilities, for example, correspondence, business keenness, and scholarly interest.

Data Engineer

Clean and quality data is crucial in the accomplishment of Big Data ventures. Consequently, we hope to see a lot of opening in 2020 for Data Engineers who have a predictable and awesome way to deal with information transformation and treatment. Organizations will search for these special data masters to have broad involvement in controlling data with SQL, T-SQL, R, Hadoop, Hive, Python and Spark. Much like Data Scientists. They are likewise expected to be innovative with regards to contrasting information with clashing information types with have the option to determine issues. They additionally frequently need to make arrangements which enable organizations to catch existing information in increasingly usable information groups – just as performing information demonstrations and their modeling.

IT/Operations Manager Job Description

In Big data industry, the IT/Operations Manager is a profitable expansion to your group and will essentially be in charge of sending, overseeing, and checking your enormous information frameworks. You’ll depend on this colleague to plan and execute new hardware and administrations. The person in question will work with business partners to comprehend the best innovation ventures to address their procedures and concerns—interpreting business necessities to innovation plans. They’ll likewise work with venture chiefs to actualize innovation and be in charge of effective progress and general activities.

Here are some responsibilities that they play:

  • Manage and be proactive in announcing, settling and raising issues where required 
  • Lead and co-ordinate issue the executive’s exercises, notwithstanding ceaseless procedure improvement activities  
  • Proactively deal with our IT framework 
  • Supervise and oversee IT staffing, including enrollment, supervision, planning, advancement, and assessment
  • Verify existing business apparatuses and procedures remain ideally practical and worth included 
  • Benchmark, dissect, report on and make suggestions for the improvement and development of the IT framework and IT frameworks 
  • Advance and keep up a corporate SLA structure

Conclusion

These are some of the best career paths that big data professionals can play after entering the industry. Honesty and hard work can always take you to the zenith of any field that you choose to be in. Also, keep upgrading your skills by taking newer certifications and technologies. Good Luck 

How can AI and Machine learning impact healthcare industry?

Healthcare industry is a recession-proof one. Even in times of economic meltdown and financial distress, the healthcare industry can hold its own because mankind will always need healthcare. In fact, during the Great Depression in the US, when the economy was facing a severe slowdown, the healthcare industry expanded, adding 852,000 jobs.

Healthcare AI in the US is slated to reach $6.6 billion in value by 2021.

From clinical trials to new drug research & development, and from innovative medical devices to technology like nanoparticles, AI, and ML has touched every point and has the power to transform them completely.

In fact, according to a study by Accenture, AI applications in healthcare can result in global savings to the tune of $150 billion by 2026.

The possibilities are endless, and the results unthinkable if AI can be properly used.

Here are some of the ways AI and ML can impact the healthcare industry:

1. Solving the Iron Triangle

A problem that has plagued the world for many years the triangle aims to solve a fundamental healthcare problem: that of good quality, accessible treatment at low cost.

Providing all three at the same time is a major challenge in healthcare, as the cost of healthcare is usually high. Here, trying to improve one factor harms another.

But AI can solve this problem in the near future without breaking the triangle, by improving the current healthcare cost-structure. The key to it is AI, and smart machines, that the patient can use for self-treatment for the majority of times, cutting down treatment costs drastically, by reducing human contact and improving quality of life.

2. Diagnostics and Imaging

The US FDA has drastically increased investment on AI in radiology and diagnostics. And it’s not without reason.

The IDx-DR became the first AI system cleared by the US FDA to provide diagnostic decisions. It was a breakthrough discovery to detect early mild diabetic retinopathy. The device was accurate 87.5% of the times, and also detected patients who didn’t have the condition, correctly up to 89.5% of times.

The US FDA also permitted marketing of the Viz.AI a type of clinical decision support system designed to analyze CT scan results to identify possibilities of a stroke in the patients and send the results to a specialist to identify any block.

In fact, diagnostics is fast becoming one of the significant drivers of AI investment in healthcare.

These advances can impact the healthcare industry in a novel way. As more and more devices become AI-enabled, the landscape of healthcare delivery will change.

3. Early screening 

Early screening in case of most diseases can drastically improve the mortality rates of patients and cut down treatment costs by over 50%.

Let’s take the example of colorectal cancer.

The 5-year survival for Stage 1 CRC is around 90%, as compared to only 10% for Stage 4.

Early detection of CRC can be ideally treated with a minimally invasive endoscopy at a low cost of less than $5,000 per year. However, in the case of late-stage CRC, it requires multidisciplinary treatment with multiple surgeries, chemotherapy, and radiation, skyrocketing the costs.

And that is why early detection is essential, and that’s exactly what AI can do. There are already apps on the market that are doing this. For example, Autism & Beyond is a revolutionary app that leveraged the power of Apple’s ResearchKit to gather videos of children and detect their preference for the development of autism, using AI software.

AI used for early screening can save billions of taxpayer dollars of taxpayer money every year, and reduce out of pocket expenditure in the US drastically.

4. Drug research & development

According to the California Biomedical Research Association, it takes around 12 years for a drug to be conceived in the laboratory and go to the patient.

Only 1 out of 5000 drugs that are selected for pre-clinical testing are then used for human testing, and only 20% of them make it to the market for human use.

(image)

The cost to develop a new drug now is more than $2.5 billion.

It is only recently that AI is being used in drug research and discovery. The power of AI can be leveraged to streamline the drug discovery and drug repurposing processes. It can identify patients best suited to the trial, can identify patients in the most need for new medications and can predict any side-effects and idiosyncrasies beforehand.

All of these, for a start, can lead to much safer clinical trials with no unwanted drug reactions.

And then, there is the question of lowering costs. In fact, a study by Carnegie Mellon and a German university estimated that AI could lower drug discovery costs by as much as 70%.

This, in turn, will be transferred to patients in the form of lower drug prices, which will increase accessibility to better medications for patients and improve population health in general.

5. Surgery 

AI-enabled robotic-assisted surgeries are taking over the US. They are increasingly being used to reduce surgeon variations and improve quality.

‘Artificial intelligence can help surgeons perform better’ quotes Dr. John Birkmeyer, a chief clinical officer at Sound Physicians.

Advanced analytics and machine learning techniques are being used concomitantly used to unleash critical insights from the billions of data elements associated with robotic-assisted surgery. If used properly, this can help overcome attendant inefficiencies and improve patient health outcomes.

Artificial intelligence helps surgeons make better clinical decisions in real-time during surgery, and helps them understand the dynamics of the patient, especially during complex operations. It also reduces the length of stay of patients by 21%.

This is ultimately reflected in the patient’s post-operative care and long-term health. It also prevents patient readmissions, saving millions of dollars annually.

A study involving 379 orthopedic patients found out that AI-assisted robotic surgery resulted in five times fewer complications as compared to surgeons working alone.

According to Accenture, AI-assisted robotic surgery could save the US healthcare industry $40 billion annually, by 2026.

6. AI-assisted virtual nurses

AI-assisted virtual nurses could well end up saving the US healthcare industry $20 billion annually, by 2026.

They are available 24/7 to answer any patient queries, monitor patients, and guide them in any way they might want.

Currently, they act as a bridge for information exchange between care providers (doctors) and care receivers (patients), to decide what medications to start, the current health status, the most recent test results, and many other things.

It can save the patient many physical appointments with doctors, and also prevent high hospital readmission rates through simple, engaging, and intelligent care.

Care Angel is one of the finest virtual nurses around. Apart from all of the above, it can also provide wellness checks through voice and AI.

Wrap-Up 

AI and ML in healthcare are still at its infancy. Adoption at a large-scale is missing as of yet. To be successful in the healthcare domain, AI and ML need the endorsement of healthcare providers like physicians and nurses.

However, considerable investment is being made in AI in healthcare, and its increasing at a good rate.

AI in healthcare is currently aimed at improving patient outcomes, taking care of the interests of various stakeholders involved, increasing accessibility, and reducing healthcare costs.

In the near future, however, AI and ML, along with technologies like Data Science will take up a much more holistic role to drive healthcare forward.

Visual Question Answering with Keras – Part 2: Making Computers Intelligent to answer from images

Making Computers Intelligent to answer from images

This is my second blog on Visual Question Answering, in the last blog, I have introduced to VQA, available datasets and some of the real-life applications of VQA. If you have not gone through then I would highly recommend you to go through it. Click here for more details about it.

In this blog post, I will walk through the implementation of VQA in Keras.

You can download the dataset from here: https://visualqa.org/index.html. All my experiments were performed with VQA v2 and I have used a very tiny subset of entire dataset i.e all samples for training and testing from the validation set.

Table of contents:

  1. Preprocessing Data
  2. Process overview for VQA
  3. Data Preprocessing – Images
  4. Data Preprocessing through the spaCy library- Questions
  5. Model Architecture
  6. Defining model parameters
  7. Evaluating the model
  8. Final Thought
  9. References

NOTE: The purpose of this blog is not to get the state-of-art performance on VQA. But the idea is to get familiar with the concept. All my experiments were performed with the validation set only.

Full code on my Github here.


1. Preprocessing Data:

If you have downloaded the dataset then the question and answers (called as annotations) are in JSON format. I have provided the code to extract the questions, annotations and other useful information in my Github repository. All extracted information is stored in .txt file format. After executing code the preprocessing directory will have the following structure.

All text files will be used for training.

 

2. Process overview for VQA:

As we have discussed in previous post visual question answering is broken down into 2 broad-spectrum i.e. vision and text.  I will represent the Neural Network approach to this problem using the Convolutional Neural Network (for image data) and Recurrent Neural Network(for text data). 

If you are not familiar with RNN (more precisely LSTM) then I would highly recommend you to go through Colah’s blog and Andrej Karpathy blog. The concepts discussed in this blogs are extensively used in my post.

The main idea is to get features for images from CNN and features for the text from RNN and finally combine them to generate the answer by passing them through some fully connected layers. The below figure shows the same idea.

 

I have used VGG-16 to extract the features from the image and LSTM layers to extract the features from questions and combining them to get the answer.

3. Data Preprocessing – Images:

Images are nothing but one of the input to our model. But as you already may know that before feeding images to the model we need to convert into the fixed-size vector.

So we need to convert every image into a fixed-size vector then it can be fed to the neural network. For this, we will use the VGG-16 pretrained model. VGG-16 model architecture is trained on millions on the Imagenet dataset to classify the image into one of 1000 classes. Here our task is not to classify the image but to get the bottleneck features from the second last layer.

Hence after removing the softmax layer, we get a 4096-dimensional vector representation (bottleneck features) for each image.

Image Source: https://www.cs.toronto.edu/~frossard/post/vgg16/

 

For the VQA dataset, the images are from the COCO dataset and each image has unique id associated with it. All these images are passed through the VGG-16 architecture and their vector representation is stored in the “.mat” file along with id. So in actual, we need not have to implement VGG-16 architecture instead we just do look up into file with the id of the image at hand and we will get a 4096-dimensional vector representation for the image.

4. Data Preprocessing through the spaCy library- Questions:

spaCy is a free, open-source library for advanced Natural Language Processing (NLP) in Python. As we have converted images into a fixed 4096-dimensional vector we also need to convert questions into a fixed-size vector representation. For installing spaCy click here

You might know that for training word embeddings in Keras we have a layer called an Embedding layer which takes a word and embeds it into a higher dimensional vector representation. But by using the spaCy library we do not have to train the get the vector representation in higher dimensions.

 

This model is actually trained on billions of tokens of the large corpus. So we just need to call the vector method of spaCy class and will get vector representation for word.

After fitting, the vector method on tokens of each question will get the 300-dimensional fixed representation for each word.

5. Model Architecture:

In our problem the input consists of two parts i.e an image vector, and a question, we cannot use the Sequential API of the Keras library. For this reason, we use the Functional API which allows us to create multiple models and finally merge models.

The below picture shows the high-level architecture idea of submodules of neural network.

After concatenating the 2 different models the summary will look like the following.

The below plot helps us to visualize neural network architecture and to understand the two types of input:

 

6. Defining model parameters:

The hyperparameters that we are going to use for our model is defined as follows:

If you know what this parameter means then you can play around it and can get better results.

Time Taken: I used the GPU on https://colab.research.google.com and hence it took me approximately 2 hours to train the model for 5 epochs. However, if you train it on a PC without GPU, it could take more time depending on the configuration of your machine.

7. Evaluating the model:

Since I have used the very small dataset for performing these experiments I am not able to get very good accuracy. The below code will calculate the accuracy of the model.

 

Since I have trained a model multiple times with different parameters you will not get the same accuracy as me. If you want you can directly download mode.h5 file from my google drive.

 

8. Final Thoughts:

One of the interesting thing about VQA is that it a completely new field. So there is absolutely no end to what you can do to solve this problem. Below are some tips while replicating the code.

  1. Start with a very small subset of data: When you start implementing I suggest you start with a very small amount of data. Because once you are ready with the whole setup then you can scale it any time.
  2. Understand the code: Understanding code line by line is very much helpful to match your theoretical knowledge. So for that, I suggest you can take very few samples(maybe 20 or less) and run a small chunk (2 to 3 lines) of code to get the functionality of each part.
  3. Be patient: One of the mistakes that I did while starting with this project was to do everything at one go. If you get some error while replicating code spend 4 to 5 days harder on that. Even after that if you won’t able to solve, I would suggest you resume after a break of 1 or 2 days. 

VQA is the intersection of NLP and CV and hopefully, this project will give you a better understanding (more precisely practically) with most of the deep learning concepts.

If you want to improve the performance of the model below are few tips you can try:

  1. Use larger datasets
  2. Try Building more complex models like Attention, etc
  3. Try using other pre-trained word embeddings like Glove 
  4. Try using a different architecture 
  5. Do more hyperparameter tuning

The list is endless and it goes on.

In the blog, I have not provided the complete code you can get it from my Github repository.

9. References:

  1. https://blog.floydhub.com/asking-questions-to-images-with-deep-learning/
  2. https://tryolabs.com/blog/2018/03/01/introduction-to-visual-question-answering/
  3. https://github.com/sominwadhwa/vqamd_floyd

Events

Nothing Found

Sorry, no posts matched your criteria