Seq2seq models and simple attention mechanism: backbones of NLP tasks

This is the second article of my article series “Instructions on Transformer for people outside NLP field, but with examples of NLP.”

1 Machine translation and seq2seq models

I think machine translation is one of the most iconic and commercialized tasks of NLP. With modern machine translation you can translate relatively complicated sentences, if you tolerate some grammatical errors. As I mentioned in the third article of my series on RNN, research on machine translation already started in the early 1950s, and their focus was translation between English and Russian, highly motivated by Cold War. In the initial phase, machine translation was rule-based, like most students do in their foreign language classes. They just implemented a lot of rules for translations. In the next phase, machine translation was statistics-based. They achieved better performance with statistics for constructing sentences. At any rate, both of them highly relied on feature engineering, I mean, you need to consider numerous rules of translation and manually implement them. After those endeavors of machine translation, neural machine translation appeared. The advent of neural machine translation was an earthshaking change of machine translation field. Neural machine translation soon outperformed the conventional techniques, and it is still state of the art. Some of you might felt that machine translation became more or less reliable around that time.

Source: Monty Python’s Life of Brian (1979)

I think you have learnt at least one foreign or classical language in school. I don’t know how good you were at the classes, but I think you had to learn some conjugations of them and I believe that was tiresome to most of students. For example, as a foreign person, I still cannot use “der”, “die”, “das” properly. Some of my friends recommended I do not care them for the time being while I speak, but I usually care grammar very much. But this method of learning language is close to the rule base machine translation, and modern neural machine translation basically does not rely on such rules.

As far as I understand, machine translation is pattern recognition learned from a large corpus. Basically no one implicitly teach computers how grammar works. Machine translation learns very complicated mapping from a source language to a target language, based on a lot of examples of word or sentence pairs. I am not sure, but this might be close to how bilingual kids learn how the two languages are related. You do not need to navigate the translator to learn specific grammatical rules.

Source: Monty Python’s Flying Circus (1969)

Since machine translation does not rely on manually programming grammatical rules, basically you do not need to prepare another specific network architecture for another pair of languages. The same method can be applied to any pairs of languages, as long as you have an enough size of corpus for that. You do not have to think about translation rules between other pairs of languages.

Source: Monty Python’s Flying Circus (1969)

*I do not follow the cutting edge studies on machine translation, so I am not sure, but I guess there are some heuristic methods for machine translation. That is, designing a network depending on the pair of languages could be effective. When it comes grammatical word orders, English and Japanese have totally different structures, I mean English is basically SVO and Japanese is basically SOV. In many cases, the structures of sentences with the same meaning in both of the languages are almost like reflections in a mirror. A lot of languages have similar structures to English, even in Asia, for example Chinese. On the other hand relatively few languages have Japanese-like structures, for example Korean, Turkish. I guess there would be some grammatical-structure-aware machine translation networks.

Not only machine translations, but also several other NLP tasks, such as summarization, question answering, use a model named seq2seq model (sequence to sequence model). As well as other deep learning techniques, seq2seq models are composed of an encoder and a decoder. In the case of seq2seq models, you use RNNs in both the encoder and decoder parts. For the RNN cells, you usually use a gated RNN such as LSTM or GRU because simple RNNs would suffer from vanishing gradient problem when inputs or outputs are long, and those in translation tasks are long enough. In the encoder part, you just pass input sentences. To be exact, you input them from the first time step to the last time step, every time giving an output, and passing information to the next cell via recurrent connections.

*I think you would be confused without some understandings on how RNNs propagate forward. You do not need to understand this part that much if you just want to learn Transformer. In order to learn Transformer model, attention mechanism, which I explain in the next section is more important. If you want to know how basic RNNs work, an article of mine should help you.

*In the encoder part of the figure below, the cell also propagate information backward. I assumed an encoder part with bidirectional RNNs, and they “forward propagate” information backwards. But in the codes below, we do not consider such complex situation. Please just keep it in mind that seq2seq model could use bidirectional RNNs.

At the last time step in the encoder part, you pass the hidden state of the RNN to the decoder part, which I show as a yellow cell in the figure below, and the yellow cell/layer is the initial hidden layer of the first RNN cell of the decoder part. Just as normal RNNs, the decoder part start giving out outputs, and passing information via reccurent connections. At every time step you choose a token to give out from the vocabulary you use in the task. That means, each cell of decoder RNNs does a classification task and decides which word to write out at the time step. Also, very importantly, in the decoder part, the output at one time step is the input at the next time step, as I show as dotted lines in the figure below.

*The translation algorithm I explained depends on greedy decoding, which has to decide a token at every time step. However it is easy to imagine that that is not how you translate a word. You usually erase the earlier words or you construct some possibilities in your mind. Actually, for better translations you would need decoding strategies such as beam search, but it is out of the scope of at least this article. Thus we are going to make a very simplified translator based on greedy decoding.

2 Learning by making

*It would take some hours on your computer to train the translator if you do not use a GPU. I recommend you to run it at first and continue reading this article.

Seq2seq models do not have that complicated structures, and for now you just need to understand the points I mentioned above. Rather than just formulating the models, I think it would be better to understand this model by actually writing codes. If you copy and paste the codes in this Github page or the official Tensorflow tutorial, installing necessary libraries, it would start training the seq2seq model for Spanish-English translator. In the Github page, I just added comments to the codes in the official tutorial so that they are more understandable. If you can understand the codes in the tutorial without difficulty, I have to say this article itself is not compatible to your level. Otherwise, I am going to help you understand the tutorial with my original figures. I made this article so that it would help you read the next article. If you have no idea what RNN is, at least the second article of my RNN series should be helpful to some extent.

*If you try to read the the whole article series of mine on RNN, I think you should get prepared. I mean, you should prepare some pieces of paper and a pen. It would be nice if you have some stocks of coffee and snacks. Though I do not think you have to do that to read this article.

2.1 The corpus and datasets

In the codes in the Github page, please ignore the part sandwiched by “######”.  Handling language data is not the focus of this article. All you have to know is that the codes below first create datasets from the Spanish-English corpus in , and you datasets for training the translator as the tensors below.

Each token is encoded with integers as the codes below, thus after encoding, the Spanish sentence “Todo sobre mi madre.” is [1, 74, 514, 19, 237, 3, 2].

2.2 The encoder

The encoder part is relatively simple. All you have to keep in mind is that you put input sentences, and pass the hidden layer of the last cell to the decoder part. To be more concrete, an RNN cell receives an input word every time step, and gives out an output vector at each time step, passing hidden states to the next cell. You make a chain of RNN cells by the process, like in the figure below. In this case “time steps” means the indexes of the order of the words. If you more or less understand how RNNs work, I think this is nothing difficult. The encoder part passes the hidden state, which is in yellow in the figure below, to the decoder part.

Let’s see how encoders are implemented in the code below. We use a type of RNN named GRU (Gated Recurrent Unit). GRU is simpler than LSTM (Long Short-Term Memory). One GRU cell gets an input every time step, and passes one hidden state via recurrent connections. As well as LSTM, GRU is a gated RNN so that it can mitigate vanishing gradient problems. GRU was invented after LSTM for smaller computation costs. At time step (t) one GRU cell gets an input \boldsymbol{x}^{(t)} and passes its hidden state/vector \boldsymbol{h}^{(t)} to the next cell like the figure below. But in the implementation, you put the whole input sentence as a 16 dimensional vector whose elements are integers, as you saw in the figure in the last subsection 2.1. That means, the ‘Encoder’ class in the implementation below makes a chain of 16 GRU cells every time you put an input sentence in Spanish, even if input sentences have less than 16 tokens.

*TO BE  VERY HONEST, I am not sure why the encoder part of  seq2seq models are implemented this way in the codes below. In the implementation below, the number of total time steps in the encoder part is fixed to 16. If input sentences have less than 16 tokens, it seems the RNN cells get no inputs after the time step of the token “<end>”. As far as I could check, if RNN cells get no inputs, they repeats giving out similar 1024-d vectors. I think in this implementation, RNN cells after the <end> token, which I showed as the dotted RNN cells in the figure above, do not change so much. And the encoder part passes the hidden state of the 16th RNN cell, which is in yellow, to the decoder.

2.3 The decoder

The decoder part is also not that hard to understand. As I briefly explained in the last section, you initialize the first cell of the decoder, using the hidden layer of the last cell the encoder. During decoding, I mean while writing a translation, at the beginning you put the token “<start>” as the first input of the decoder. Given the input “<start>”, the first cell outputs “all” in the example in the figure below, and the output “all” is the input of the next cell. The output of the next cell “about” is also passed to the next cell, and you repeat this till the decoder gives out the token “<end>”.

A more important point is how to get losses in the decoder part during training. We use a technique named teacher enforcing during training the decoder part of a seq2seq model. This is also quite simple: you just have to make sure you input a correct answer to RNN cells, regardless of the outputs generated by the cell last time step. You force the decoder to get the correct input every time step, and that is what teacher forcing is all about.

You can see how the decoder part and teacher forcing is implemented in the codes below. You have to keep it in mind that unlike the ‘Encoder’ class, you put a token into a ‘Decoder’ class every time step. To be exact you also need the outputs of the encoder part to calculate attentions in the decoder part. I am going to explain that in the next subsection.

2.4 Attention mechanism

I think you have learned at least one foreign language, and usually you have to translate some sentences. Remember the processes of writing a translation of a sentence in another language. Imagine that you are about to write a new word after writing some. If you are not used to translations in the language, you must have cared about which parts of the original language correspond to the very new word you are going to write. You have to pay “attention” to the original sentence. This is what attention mechanism is all about.

*I would like you to pay “attention” to this section. As you can see from the fact that the original paper on Transformer model is named “Attention Is All You Need,” attention mechanism is a crucial idea of Transformer.

In the decoder part you initialize the hidden layer with the last hidden layer of the encoder, and its first input is “<start>”.  The decoder part start decoding, , as I explained in the last subsection. If you use attention mechanism in the seq2seq model, you calculate attentions every times step.  Let’s consider an example in the figure below, where the next input in the decoder is “my”, and given the token “my”, the GRU cell calculates a hidden state at the time step. The hidden state is the “query” in this case, and you compare the “query” with the 6 outputs of the encoder, which are “keys”. You get weights/scores, I mean “attentions”, which is the histogram in the figure below.

And you reweight the “values” with the weights in the histogram. In this case the “values” are the outputs of the encoder themselves. You used use the reweighted “values” to calculate the hidden state of the decoder at the times step again. And you used the hidden state updated by the attentions to predict the next word.

*In the implementation, however, the size of the output of the ‘Encoder’ class is always (16, 2024). You calculate attentions for all those 16 output vectors, but virtually only the first 6 1024-d output vectors important.

Summing up the points I have explained, you compare the “query” with the “keys” and get scores/weights for the “values.” Each score/weight is in short the relevance between the “query” and each “key”. And you reweight the ‘values’ with the scores/weights, and take the summation of the reweighted “values.” In the case of attention mechanism in this article, we can say that “values” and “keys” are the same. You would also see that more clearly in the implementation below.

You especially have to pay attention to the terms “query”, “key”, and “value.” “Keys” and “values” are basically in the same language, and in the case above, they are in Spanish. “Queries” and “keys” can be in either different or the same. In the example above, the “query” is in English, and the “keys” are in Spanish.

You can compare a “query” with “keys” in various ways. The implementation uses the one called  Bahdanau’s additive style, and in Transformer, you use more straightforward ways. You do not have to care about how Bahdanau’s additive style calculates those attentions. It is much more important to learn the relations of “queries”, “keys”, and “values” for now.

*A problem is that Bahdanau’s additive style is slightly different from the figure above. It seems in Bahdanau’s additive style, at the time step (t) in the decoder part, the query is the hidden state at the time step (t-1). You would notice that if you closely look at the implementation below.As you can see in the figure above, you can see that you have to calculate the hidden state of the decoder cell two times at the time step (t): first in order to generate a “query”, second in order to predict the translated word at the time step. That would not be so computationally efficient, and I guess that is why Bahdanau’s additive style uses the hidden layer at the last time step as a query rather than calculating hidden layers twice.

2.5 Translating and displaying attentions

After training the translator for 20 epochs, I could translate Spanish sentences, and the implementation also displays attention scores for between the input and output sentences. For example the translation of the inputs “Todo sobre mi madre.” and “Habre con ella.” were “all about my mother .” and “i talked to her .” respectively, and the results seem fine. One powerful advantage of using attention mechanism is you can display this type of word alignment, I mean correspondences of words in a sentence, easily as in the heat maps below. The yellow parts shows high scores of attentions, and you can see that the distributions of relatively highs scores are more or less diagonal, which implies that English and Spanish have similar word orders.

For other inputs like “Mujeres al borde de un ataque de nervious.” or “Volver.”, the translations are not good.

You might have noticed there is one big problem in this implementation: you can use only the words appeared in the corpus. And actually I had to manually add some pairs of sentences with the word “borde” to the corpus to get the translation in the figure.


[1] “Neural machine translation with attention,” Tensorflow Core

[2]Tsuboi Yuuta, Unno Yuuya, Suzuki Jun, “Machine Learning Professional Series: Natural Language Processing with Deep Learning,” (2017), pp. 72-85, 91-94
坪井祐太、海野裕也、鈴木潤 著, 「機械学習プロフェッショナルシリーズ 深層学習による自然言語処理」, (2017), pp. 72-85, 191-193

[3]”Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 8 – Translation, Seq2Seq, Attention”, stanfordonline, (2019)

* I make study materials on machine learning, sponsored by DATANOMIQ. I do my best to make my content as straightforward but as precise as possible. I include all of my reference sources. If you notice any mistakes in my materials, including grammatical errors, please let me know (email: And if you have any advice for making my materials more understandable to learners, I would appreciate hearing it.

AI Voice Assistants are the Next Revolution: How Prepared are You?

By 2022, voice-based shopping is predicted to rise to USD 40 billion, based on the data from OC&C Strategy Consultants. We’re in an era of ‘voice’ where drastic transformation is seen between the way AI and voice recognition are changing the way we live.

According to the survey, the surge of voice assistants is said to be driven by the number of homes that used smart speakers, as such that the rise is seen to grow from 13% to 55%. Nonetheless, Amazon will be one of the leaders to dominate the new channel having the largest market share.

Perhaps this is the first time you’ve heard about the voice revolution. Well, why not, based on multiple researchers, it is estimated that the number of voice assistants will grow to USD 8 billion by 2023 from USD 2.5 billion in 2018.

But what is voice revolution or voice assistant or voice search?

It was only until recently that the consumers have started learning about voice assistants which further predicts to exist in the future.

You’ve heard of Alexa, Cortana, Siri, and Google Assistant, these technologies are some of the world’s greatest examples of voice assistants. They will further help to drive consumer behavior as well as prepare the companies and adjust based on the industry demands. Consumers can now transform the way they act, search, and advertise their brand through voice technology.

Voice search is a technology to help users or consumers perform a search on the website by simply asking a question on their smartphone, their computer, or their smart device.

The voice assistant awareness: Why now?

As surveyed by PwC, amongst the 90% respondents, about 72% have been recorded to use voice assistant while merely 10% said they were clueless about voice-enabled devices and products. It is noted, the adoption of voice-enabled was majorly driven by children, young consumers, and households earning an income of around >USD100k.

Let us have a glance to ensure the devices that are used mainly for voice assistance: –

  • Smartphone – 57%
  • Desktop – 29%
  • Tablet – 29%
  • Laptop – 29%
  • Speaker – 27%
  • TV remote – 21%
  • Car navigation – 20%
  • Wearable – 14%

According to the survey, most consumers that use voice-assistants were the younger generation, aged between 18-24.

While individuals between the ages 25-49 were said to use these technologies in a much more statistical manner, and are called the “heavy users.”

Significance of mobile voice assistants: What is the need?

Although mobile is accessible everywhere, you will merely find three out of four consumers using mobile voice assistants in their household i.e. 74%.

Mobile-based AI chatbots have taken our lives by storm, thus providing the best solution to both the customers and agents in varied areas – insurance, travel, and education, etc.

A certain group of individuals said they needed privacy while speaking to their device and that sending a voice command in public is weird.

Well, this simply explains why 18-24 aged group individuals prefer less use of voice assistants. However, this age group tends to spend more time out of their homes.

Situations where voice assistants can be used – standalone speakers Vs mobile


  • Standalone speakers – 65%
  • Mobile – 37%


  • Standalone speakers – 62%
  • Mobile – 12%

Watching TV

  • Standalone speakers – 57%
  • Mobile – 43%

In bed

  • Standalone speakers – 38%
  • Mobile – 37%


  • Standalone speakers – 29%
  • Mobile – 25%


  • Standalone speakers – 0%
  • Mobile – 40%

By the end of 2020, nearly half of all the searches made will be voice-based, as predicted by Comscore, a media analytics firm.

Don’t you think voice-based assistant is changing the way businesses function? Thanks to the advent of AI!

  • A 2018 study on AI chatbots and voice assistants by Spiceworks said, 24% of businesses that were spread largely, and 16% of smaller businesses have already started using AI technologies in their workplaces. While 25% of the business market is expected to adopt AI within the next 12 months.

Surprisingly, voice-based assistants such as Siri, Google Assistant, and Cortana are some of the most prominent technologies these businesses are using in their workstations.

Where will the next AI voice revolution take us?

Voice-authorized transactions

Paypal, an online payment gateway now leverages Siri and Alexa’s voice recognition capability, thus, allowing users to make payments, check their balance, and ask payments from people via voice command.

Voice remote control – AI-powered

Communications conglomerate Comcast, an American telecommunications and media conglomerate introduces their first-ever X1 voice remote control that provides both natural image processing and voice recognition.

With the help of deep learning, the X1 can easily come up with better search results with just a press of the button telling what your television needs to do next.

Voice AI-enabled memos and analytics

Salesforce recently unveiled Einstein Voice which is an AI assistant that helps in entering critical data the moment it hears, making use of the voice command. This AI assistant also initiates in interpreting voice memos. Besides this, the voice bots accompanying Einstein Voice also helps the company create their customized voice bots to answer customer queries.

Voice-activated ordering

It is astonishing to see how Domino’s is using voice-activated feature automate orders made over the phone by customers. Well, welcome to the era of voice revolution.

This app, developed by Nuance Communications already has a Siri like voice recognition feature that allows customers to place their orders just like how they would be doing it in front of the cash counter making your order to take place efficiently.

As more businesses look forward to breaking down the roadblocks between a consumer and a brand, voice search now projects to become an impactful technology of bridging the gap.

Simple RNN

A gentle introduction to the tiresome part of understanding RNN

Just as a normal conversation in a random pub or bar in Berlin, people often ask me “Which language do you use?” I always answer “LaTeX and PowerPoint.”

I have been doing an internship at DATANOMIQ and trying to make straightforward but precise study materials on deep learning. I myself started learning machine learning in April of 2019, and I have been self-studying during this one-year-vacation of mine in Berlin.

Many study materials give good explanations on densely connected layers or convolutional neural networks (CNNs). But when it comes to back propagation of CNN and recurrent neural networks (RNNs), I think there’s much room for improvement to make the topic understandable to learners.

Many study materials avoid the points I want to understand, and that was as frustrating to me as listening to answers to questions in the Japanese Diet, or listening to speeches from the current Japanese minister of the environment. With the slightest common sense, you would always get the feeling “How?” after reading an RNN chapter in any book.

This blog series focuses on the introductory level of recurrent neural networks. By “introductory”, I mean prerequisites for a better and more mathematical understanding of RNN algorithms.

I am going to keep these posts as visual as possible, avoiding equations, but I am also going to attach some links to check more precise mathematical explanations.

This blog series is composed of five contents.:

  1. Prerequisites for understanding RNN at a more mathematical level
  2. Simple RNN: the first foothold for understanding LSTM
  3. A brief history of neural nets: everything you should know before learning LSTM
  4. Understanding LSTM forward propagation in two ways
  5. LSTM back propagation: following the flows of variables


Business Data is changing the world’s view towards Green Energy

Energy conservation is one of the main stressed points all around the globe. In the past 30 years, researches in the field of energy conservation and especially green energy have risen to another level. The positive outcomes of these researches have given us a gamut of technologies that can aid in preserving and utilize green energy. It has also reduced the over-dependency of companies on fossil fuels such as oil, coal, and natural gas.

Business data and analytics have all the power and the potential to take the business organizations forward in the future and conquer new frontiers. Seizing the opportunities presented by Green energy, market leaders such as Intel and Google have already implemented it, and now they enjoy the rich benefits of green energy sources.

Business data enables the organizations to keep an eye on measuring the positive outcomes by adopting the green energies. According to a report done by the World energy outlook, the global wind energy capacity will increase by 85% by the year 2020, reaching 1400 TWh. Moreover, in the Paris Summit, more than 170 countries around the world agreed on reducing the impact of global warming by harnessing energy from green energy sources. And for this to work, Big Data Analytics will play a pivotal role.

Overview of Green energy

In simpler terms, Green Energy is the energy coming from natural sources such as wind, sun, plants, tides, and geothermal heat. In contrast to fossil fuels, green energy resources can be replenished in a short period, and one can use them for longer periods. Green energy sources have a minimal ill effect on the environment as compared to fossil fuels. In addition to this, fossil fuels can be replaced by green energy sources in many areas like providing electricity, fuel for motor vehicles, etc..

With the help of business data, organizations throughout the world can change the view of green energy. Big Data can show how different types of green energy sources can help businesses and accelerate sustainable expansion.

Below are the different types of green energy sources:

  • Wind Power
  • Solar Power
  • Geothermal Energy
  • Hydropower
  • Biofuels
  • Bio-mass

Now we present before you a list of advantages that green energy or renewable energy sources have brought to the new age businesses.

Profits on the rise

If the energy produced is more than the energy used, the organizations can sell it back to the grids and earn profit out of it. Green energy sources are renewable sources of energy, and with precise data, the companies will get an overall estimation of the requirement of energy.

With Big Data, the organizations can know the history of the demographical location before setting up the factory. For example, if your company is planning to setup a factory in the coastal region, tidal and wind energy would be more beneficial as compared to solar power. Business data will give the complete analysis of the flow of the wind so that the companies can ascertain the best location of the windmill; this will allow them to store the energy in advance and use it as per their requirement. It not only saves money but also provides an extra source of income to the companies. With green energy sources, the production in the company can increase to an unprecedented level and have sustainable growth over the years.

Synchronizing the maintenance process

If there is a rapid inflow of solar and wind energy sources, the amount of power produced will be huge. Many solar panels and windmills are operating in a solar power plant or in a wind energy source, and with many types of equipment, it becomestoo complex to manage. Big Data analytics will assist the companies in streamlining all the operations to a large extent for their everyday work without any hassle.

Moreover, the analytics tool will convey the performance of renewable energy sources under different weather conditions. Thus, the companies will get the perfect idea about the performance of the green energy sources, thus enabling them to take necessary actions as and when required.

Lowering the attrition rate

Researchers have found that more number of employees want to be associated with companies that support green energies. By opting for green energy sources and investing in them, companies are indirectly investing in keeping the workforce intact and lowering the attrition rate. Stats also show the same track as nearly 50% of the working professionals, and almost 2/3rd of the millennial population want to be associated with the companies who are opting for the green energy sources and have a positive impact on environmental conservation.

The employees will not only wish to stay with the organizations for a long time but will also work hard for the betterment of the organization. Therefore, you can concentrate on expanding the business rather than thinking about the replacement of the employees.

Lowering the risk due to Power Outage

The Business Data Analytics will continuously keep updating the requirements of power needed to run the company. Thus the organizations can cut down the risk of the power outage and also the expenses related to it. The companies will know when to halt the energy transmission as they would know if the grid is under some strain or not.

Business analytics and green energy provide a planned power outage to the companies, which is cost-efficient and thus can decrease the product development cost.  Apart from this, companies can store energy for later usage. Practicing this process will help save a lot of money in the long run, proving that investment in green energy sources is a smart investment.

Reducing the maintenance cost

An increasing number of organizations are using renewable sources of energy as it plays a vital role in decreasing production and maintenance costs. The predictive analysis technology helps renewable energy sources to produce more energy at less cost, thus reducing the cost of infrastructure.

Moreover, data analytics will make green energy sources more bankable for companies. As organizations will have a concrete amount of data related to the energy sources, they can use it wisely on a more productive basis

Escalating Energy Storage

Green energy sources can be stored in bulk and used as per requirement by the business organizations. Using green energy on a larger basis will even allow companies to completely get rid of fossil fuels and thus work towards the betterment of the environment. Big Data analytics with AI and cloud-enabled systems help organizations store renewable energies such as Wind and Solar.

Moreover, it gathers information for the businesses and gives the complete analysis of the exact amount of energy required to complete a particular task. The data will also automate cost savings as it can predict the client’s needs. Based on business data, companies can store renewable energy sources in a better manner.

With Business data analytics, the companies can store energy when it is cheap and use it according to the needs when the energy rates go higher. Although predicting the requirement of storage is a complicated process, with Artificial Intelligence (AI) at work, you can analyze the data efficiently.

Bundling Up

Green energy sources will play a pivotal role in deciding the future of the businesses as fossil fuels are available in a certain limit. Moreover, astute business data analysts will assist the organizations to not only use renewable energy sources in a better manner but also to form a formidable workforce. The data support in the green energy sector will also provide sustainable growth to the companies, monitor their efforts, and assist them in the long run.

Predictive Analytics World 2020 Healthcare

Difficult times call for creative measures

Predictive Analytics World for Healthcare will go virtual and you still have time to join us!

What do you have in store for me?

We will provide a live-streamed virtual version of healthcare Munich 2020 on 11-12 May, 2020: you will be able to attend sessions and to interact and connect with the speakers and fellow members of the data science community including sponsors and exhibitors from your home or your office.

What about the workshops?

The workshops will also be held virtually on the planned date:
13 May, 2020.

Get a complimentary virtual sneak preview!

If you would like to join us for a virtual sneak preview of the workshop „Data Thinking“ on Thursday, April 16, so you can familiarise yourself with the quality of the virtual edition of both conference and workshops and how the interaction with speakers and attendees works, please send a request to

Don’t have a ticket yet?

It‘s not too late to join the data science community.
Register by 10 May to receive access to the livestream and recordings.


We’re looking forward to see you – virtually!

This year Predictive Analytics World for Healthcare runs alongside Deep Learning World and Predictive Analytics World for Industry 4.0.

Customer Journey Mapping: The data-driven approach to understanding your users

Businesses across the globe are on a mission to know their customers inside out – something commonly referred to as customer-centricity. It’s an attempt to better understand the needs and wants of customers in order to provide them with a better overall experience.

But while this sounds promising in theory, it’s much harder to achieve in practice. To really know your customer you must not only understand what they want, but you also need to hone in on how they want it, when they want it and how often as well.

In essence, your business should use customer journey mapping. It allows you to visualise customer feelings and behaviours through the different stages of their journey – from the first interaction, right up until the point of purchase and beyond.

The Data-Driven Approach 

To ensure your customer journey mapping is successful, you must conduct some extensive research on your customers. You can’t afford to make decisions based on feelings and emotions alone. There are two types of research that you should use for customer journey mapping – quantitative and qualitative research.

Quantitative data is best for analysing the behaviour of your customers as it identifies their habits over time. It’s also extremely useful for confirming any hypotheses you may have developed. That being so, relying solely upon quantitative data can present one major issue – it doesn’t provide you with the specific reason behind those behaviours.

That’s where qualitative data comes to the rescue. Through data collection methods like surveys, interviews and focus groups, you can figure out the reasoning behind some of your quantitative data trends. The obvious downside to qualitative data is its lack of evidence and its tendency to be subjective. Therefore, a combination of both quantitative and qualitative research is most effective.

Creating A Customer Persona

A customer persona is designed to help businesses understand the key traits of specific groups of people. For example, those defined by their age range or geographic location. A customer persona can help improve your customer journey map by providing more insight into the behavioural trends of your “ideal” customer. 

The one downside to using customer personas is that they can be over-generalised at times. Just because a group of people shares a similar age, for example, it does not mean they all share the same beliefs and interests. Nevertheless, creating a customer persona is still beneficial to customer journey mapping – especially if used in combination with the correct customer journey analytics tools.

All Roads Lead To Customer-centricity 

To achieve customer-centricity, businesses must consider using a data-driven approach to customer journey mapping. First, it requires that you achieve a balance between both quantitative and qualitative research. Quantitative research will provide you with definitive trends while qualitative data gives you the reasoning behind those trends. 

To further increase the effectiveness of your customer journey map, consider creating customer personas. They will give you further insight into the behavioural trends within specific groups. 

This article was written by TAP London. Experts in the Adobe Experience Cloud, TAP London help brands organise data to provide meaningful insight and memorable customer experiences. Find out more at

5 Applications for Location-Based Data in 2020

Location-based data enables giving people relevant information based on where they are at any given moment. Here are five location data applications to look for in 2020 and beyond. 

1. Increasing Sales and Reducing Frustration

One 2019 report indicated that 89% of the marketers who used geo data saw increased sales within their customer bases. Sometimes, the ideal way to boost sales is to convert what would be a frustration into something positive. 

A French campaign associated with the Actimel yogurt brand achieved this by sending targeted, encouraging messages to drivers who used the Waze navigation app and appeared to have made a wrong turn or got caught in traffic. 

For example, a driver might get a message that said, “Instead of getting mad and honking your horn, pump up the jams! #StayStrong.” The three-month campaign saw a 140% increase in ad recall. 

More recently, home furnishing brand IKEA launched a campaign in Dubai where people can get free stuff for making a long trip to a store. The freebies get more valuable as a person’s commute time increases. The catch is that participants have to activate location settings on their phones and enable Google Maps. Driving five minutes to a store got a person a free veggie hot dog, and they’d get a complimentary table for traveling 49 minutes. 

2. Offering Tailored Ad Targeting in Medical Offices

Pharmaceutical companies are starting to rely on companies that send targeted ads to patients connected to the Wi-Fi in doctors’ offices. One such provider is Semcasting. A recent effort involved sending ads to cardiology offices for a type of drug that lowers cholesterol levels in the blood. 

The company has taken a similar approach for an over-the-counter pediatric drug and a medication to relieve migraine headaches, among others. Such initiatives cause a 10% boost in the halo effect, plus a 1.5% uptick in sales. The first perk relates to the favoritism that people feel towards other products a company makes once they like one of them.

However, location data applications related to health care arguably require special attention regarding privacy. Patients may feel uneasy if they believe that companies are watching them and know they need a particular kind of medical treatment. 

3. Facilitating the Deployment of the 5G Network

The 5G network is coming soon, and network operators are working hard to roll it out. Statistics indicate that the 5G infrastructure investment will total $275 billion over seven years. Geodata can help network brands decide where to deploy 5G connectivity first.

Moreover, once a company offers 5G in an area, marketing teams can use location data to determine which neighborhoods to target when contacting potential customers. Most companies that currently have 5G within their product lineups have carefully chosen which areas are at the top of the list to receive 5G, and that practice will continue throughout 2020. 

It’s easy to envision a scenario whereby people can send error reports to 5G providers by using location data. For example, a company could say that having location data collection enabled on a 5G-powered smartphone allows a technician to determine if there’s a persistent problem with coverage.

Since the 5G network is still, it’s impossible to predict all the ways that a telecommunications operator might use location data to make their installations maximally profitable. However, the potential is there for forward-thinking brands to seize. 

4. Helping People Know About the Events in Their Areas

SoundHound, Inc. and Wcities recently announced a partnership that will rely on location-based data to keep people in the loop about upcoming local events. People can use a conversational intelligence platform that has information about more than 20,000 cities around the world. 

Users also don’t need to mention their locations in voice queries. They could say, for example, “Which bands are playing downtown tonight?” or “Can you give me some events happening on the east side tomorrow?” They can also ask something associated with a longer timespan, such as “Are there any wine festivals happening this month?”

People can say follow-up commands, too. They might ask what the weather forecast is after hearing about an outdoor event they want to attend. The system also supports booking an Uber, letting people get to the happening without hassles. 

5. Using Location-Based Data for Matchmaking

In honor of Valentine’s Day 2020, students from more than two dozen U.S colleges signed up for a matchmaking opportunity. It, at least in part, uses their location data to work. 

Participants answer school-specific questions, and their responses help them find a friend or something more. The platform uses algorithms to connect people with like-minded individuals. 

However, the company that provides the service can also give a breakdown of which residence halls have the most people taking part, or whether people generally live off-campus. This example is not the first time a university used location data by any means, but it’s different from the usual approach. 

Location Data Applications Abound

These five examples show there are no limits to how a company might use location data. However, they must do so with care, protecting user privacy while maintaining a high level of data quality. 

5 Things You Should Know About Data Mining

The majority of people spend about twenty-four hours online every week. In that time they give out enough information for big data to know a lot about them. Having people collecting and compiling your data might seem scary but it might have been helpful for you in the past.


If you have ever been surprised to find an ad targeted toward something you were talking about earlier or an invention made based on something you were googling, then you already know that data mining can be helpful. Advanced education in data mining can be an awesome resource, so it may pay to have a personal tutor skilled in the area to help you understand. 


It is understandable to be unsure of a system that collects all of the information online so that they can learn more about you. Luckily, so much data is put out every day it is unlikely data mining is focusing on any of your important information. Here are a few statistics you should know about mining.


1. Data Mining Is Used In Crime Scenes

Using a variation of earthquake prediction software and data, the Los Angeles police department and researchers were able to predict crime within five hundred feet. As they learn how to compile and understand more data patterns, crime detecting will become more accurate.


Using their data the Los Angeles police department was able to stop thief activity by thirty-three percent. They were also able to predict violent crime by about twenty-one percent. Those are not perfect numbers, but they are better than before and will get even more impressive as time goes on. 


The fact that data mining is able to pick up on crime statistics and compile all of that data to give an accurate picture of where crime is likely to occur is amazing. It gives a place to look and is able to help stop crime as it starts.


2. Data Mining Helps With Sales

A great story about data mining in sales is the example of Walmart putting beer near the diapers. The story claims that through measuring statistics and mining data it was found that when men purchase diapers they are also likely to buy a pack of beer. Walmart collected that data and put it to good use by putting the beer next to the diapers.


The amount of truth in that story/example is debatable, but it has made data mining popular in most retail stores. Finding which products are often bought together can give insight into where to put products in a store. This practice has increased sales in both items immensely just because people tend to purchase items near one another more than they would if they had to walk to get the second item. 


Putting a lot of stock in the data-gathering teams that big stores build does not always work. There have been plenty of times when data teams failed and sales plummeted. Often, the benefits outweigh the potential failure, however, and many stores now use data mining to make a lot of big decisions about their sales.


3. It’s Helping With Predicting Disease 


In 2009 Google began work to be able to predict the winter flu. Google went through the fifty million most searched words and then compared them with what the CDC was finding during the 2003-2008 flu seasons. With that information google was able to help predict the next winter flu outbreak even down to the states it hit the hardest. 


Since 2009, data mining has gotten much better at predicting disease. Since the internet is a newer invention it is still growing and data mining is still getting better. Hopefully, in the future, we will be able to predict disease breakouts quickly and accurately. 


With new data mining techniques and research in the medical field, there is hope that doctors will be able to narrow down problems in the heart. As the information grows and more data is entered the medical field gets closer to solving problems through data. It is something that is going to help cure diseases more quickly and find the root of a problem.


4. Some Data Mining Gets Ignored

Interestingly, very little of the data that companies collect from you is actually used. “Big data Companies” do not use about eighty-eight percent of the data they have. It is incredibly difficult to use all of the millions of bits of data that go through big data companies every day.


The more people that are used for data mining and the more data companies are actually able to filter through, the better the online experience will be. It might be a bit frightening to think of someone going through what you are doing online, but no one is touching any of the information that you keep private. Big data is using the information you put out into the world and using that data to come to conclusions and make the world a better place.


There is so much information being put onto the internet at all times. Twenty-four hours a week is the average amount of time a single person spends on the internet, but there are plenty of people who spend more time than that. All of that information takes a lot of people to sift through and there are not enough people in the data mining industry to currently actually go through the majority of the data being put online.


5. Too Many Data Mining Jobs

Interestingly, the data industry is booming. In general, there are an amazing amount of careers opening on the internet every day. The industry is growing so quickly that there are not enough people to fill the jobs that are being created.


The lack of talent in the industry means there is plenty of room for new people who want to go into the data mining industry. It was predicted that by 2018 there would be a shortage of 140,000 with deep analytical skills. With the lack of jobs that are being discussed, it is amazing that there is such a shortage in the data industry. 


If big data is only able to wade through less than half of the data being collected then we are wasting a resource. The more people who go into an analytics or computer career the more information we will be able to collect and utilize. There are currently more jobs than there are people in the data mining field and that needs to be corrected.


To Conclude

The data mining industry is making great strides. Big data is trying to use the information they collect to sell more things to you but also to improve the world. Also, there is something very convenient about your computer knowing the type of things you want to buy and showing you them immediately. 


Data mining has been able to help predict crime in Los Angeles and lower crime rates. It has also helped companies know what items are commonly purchased together so that stores can be organized more efficiently. Data mining has even been able to predict the outbreak of disease down to the state.


Even with so much data being ignored and so many jobs left empty, data mining is doing incredible things. The entire internet is constantly growing and the data mining is growing right along with it. As the data mining industry climbs and more people find their careers mining data the more we will learn and the more facts we will find.


Artikelserie: BI Tools im Vergleich – Tableau

Dies ist ein Artikel der Artikel-Serie “BI Tools im Vergleich – Einführung und Motivation“. Solltet ihr gerade erst eingestiegen sein, dann schaut euch ruhig vorher einmal die einführenden Worte und die Ausführungen zur Datenbasis an. Power BI machte den Auftakt und ihr findet den Artikel hier.


Tableau stellt seinen Kunden zu allererst vor die Wahl, wo und von wem die Infrastruktur betrieben werden soll. Einen preislichen Vorteil hat der Kunde bei der Wahl einer selbstverwaltenden Lösung unter Nutzung von Tableau Server. Die Alternative ist eine Cloud-Lösung, bereitgestellt und verwaltet von Tableau. Bei dieser Variante wird Tableau Server durch Tableau Online ersetzt, wobei jede dieser Optionen die gleichen Funktionalitäten mit sich bringen. Bereits das Lizenzmodell definiert unterschiedliche Rollen an Usern, welche in drei verschiedene Lizenztypen unterteilt und unterschiedlich bepreist sind (siehe Grafik). So kann der User die Rolle eines Creators, Explorers oder Viewers einnehmen.Der Creator ist befähigt, alle Funktionen von Tableau zu nutzen, sofern ein Unternehmen die angebotenen Add-ons hinzukauft. Die Lizenz Explorer ermöglicht es dem User, durch den Creator vordefinierte Datasets in Eigenregie zu analysieren und zu visualisieren. Demnach obliegt dem Creator, und somit einer kleinen Personengruppe, die Datenbereitstellung, womit eine Single Source of Truth garantiert werden soll. Der Viewer hat nur die Möglichkeit Berichte zu konsumieren, zu teilen und herunterzuladen. Wobei in Bezug auf Letzteres der Viewer limitiert ist, da dieser nicht die kompletten zugrundeliegenden Daten herunterladen kann. Lediglich eine Aggregation, auf welcher die Visualisierung beruht, kann heruntergeladen werden. Ein Vergleich zeigt die wesentlichen Berechtigungen je Lizenz.

Der Einstieg bei Tableau ist für Organisationen nicht unter 106 Lizenzen (100 Viewer, 5 Explorer, 1 Creator) möglich, und Kosten von mindestens $1445 im Monat müssen einkalkuliert werden.

Wie bereits erwähnt, existieren Leistungserweiterungen, sogennante Add-ons. Die selbstverwaltende Alternative unter Nutzung von Tableau Server (hosted by customer) kann um das Tableau Data Management Add‑on und das Server Management Add‑on erweitert werden. Hauptsächlich zur Serveradministration, Datenverwaltung und -bereitstellung konzipiert sind die Features in vielen Fällen entbehrlich. Für die zweite Alternative (hosted by Tableau) kann der Kunde ebenfalls das Tableau Data Management Add‑on sowie sogenannte Resource Blocks dazu kaufen. Letzteres lässt bereits im Namen einen kapazitätsabhängigen Kostenfaktor vermuten, welcher zur Skalierung dient. Die beiden Add‑ons wiederum erhöhen die Kosten einer jeden Lizenz, was erhebliche Kostensteigerungen mit sich bringen kann. Das Data Management Add‑on soll als Beispiel die Kostenrelevanz verdeutlichen. Es gelten $5,50 je Lizenz für beide Hosting Varianten. Ein Unternehmen bezieht 600 Lizenzen (50 Creator, 150 Explorer und 400 Viewer) und hosted Tableau Server auf einer selbstgewählten Infrastruktur. Beim Zukauf des Add‑ons erhöht sich die einzelne Viewer-Lizenz bei einem Basispreis von $12 um 46%. Eine nicht unrelevante Größe bei der Vergabe neuer Viewer-Lizenzen, womit sich ein jedes Unternehmen mit Wachstumsambitionen auseinandersetzen sollte. Die Gesamtkosten würden nach geschilderter Verteilung der Lizenzen um 24% steigen (Anmerkung: eventuelle Rabatte sind nicht mit einbezogen). Die Tatsache, dass die Zuschläge für alle Lizenzen gelten, kann zumindest kritisch hinterfragt werden.

Ein weiterer, anfangs oft unterschätzter Kostenfaktor ist die Anzahl der Explorer-Lizenzen. Das Verhältnis der Explorer-Lizenzen an der Gesamtanzahl wächst in vielen Fällen mittelfristig nach der Einführungsphase stark an. Häufig wird Tableau als eine neue State of the Art Reporting Lösung mit schönen bunten Bildern betrachtet und dessen eigentliche Stärke, die Generierung von neuen Erkenntnissen mittels Data Discovery, wird unterschätzt. Hier kommt die Explorer Lizenz ins Spiel, welche ca. das Dreifache einer Viewer Lizenz kostet und den User befähigt, tiefer in die Daten einzusteigen.

Nichtdestotrotz kann man behaupten, dass das Lizenzmodell sehr transparent ist. Tableau selbst wirbt damit, dass keine versteckten Kosten auf den Kunden zukommen. Das Lizenzmodell ist aber nicht nur auf die Endkunden ausgerichtet, sondern bietet mit Tableau Server auch ein besonders auf Partner ausgerichtetes Konzept an. Serviceanbieter können so Lizenzen erwerben und in das eigene Angebot zu selbst gewählten Konditionen aufnehmen. Eine Server Instanz reicht aus, da das Produkt auch aus technischer Sicht mit sogenannten Sites auf verschiedene Stakeholder ausgerichtet werden kann.

Community & Features von anderen Entwicklern

Die Bedeutung einer breiten Community soll hier noch einmal hervorgehoben werden. Für Nutzer ist der Austausch über Probleme und Herausforderungen sowie technischer und organisatorischer Art äußerst wichtig, und auch der Softwarehersteller profitiert davon erheblich. Nicht nur, dass der Support teilweise an die eigenen Nutzer abgegeben wird, auch kann der Anbieter bestehende Features zielgerichteter optimieren und neue Features der Nachfrage anpassen. Somit steht die Tableau Community der Power BI Community in nichts nach. Zu den meisten Themen wird man schnell fündig in diversen Foren wie auch auf der Tableau Webseite. Es existiert die klassische Community Plattform, aber auch eine Tableau Besonderheit: Tableau Public. Es handelt sich hierbei um eine kostenlose Möglichkeit eine abgespeckte Version von Tableau zu nutzen und Inhalte auf der gleichnamigen Cloud zu veröffentlichen. Ergänzend sind etliche Lernvideos auf den einschlägigen Seiten fast zu jedem Thema zu finden und komplettieren das Support-Angebot.

Zusätzlich bietet Tableau sogenannte Admin-Tools aus eigenem Hause an, welche als Plug ins eingebunden werden können. Tableau unterscheidet dabei zwischen Community Supported Tools (z.B. TabMon) und Tableau Supported Tools (z.B. Tabcmd).

Ebenfalls bietet Tableau seit der Version 2018.2 dritten Entwicklern eine sogenannte Extensions API an und ermöglicht diesen damit, auf Basis der Tableau-Produkte eigene Produkte zu entwickeln. Erst kürzlich wurde mit Sandboxed Extensions in der Version 2019.4 ein wesentlicher Schritt hin zu einer höheren Datensicherheit gemacht, so dass es zukünftig zwei Gruppen von Erweiterungen geben wird. Die erste und neue Gruppe Sandboxed Extensions beinhaltet alle Erweiterungen, bei denen die Daten das eigene Netzwerk bzw. die Cloud nicht verlassen. Alle übrigen Erweiterungen werden in der zweiten Gruppe Network-Enabled Extensions zusammengefasst. Diese kommunizieren wie gehabt mit der Außenwelt, um den jeweiligen Service bereitzustellen.

Grundsätzlich ist Tableau noch zurückhaltend, wenn es um Erweiterungen des eigenen Produktportfolios geht. Deshalb ist die Liste mit insgesamt 37 Erweiterungen von 19 Anbietern noch recht überschaubar.

Daten laden & transformieren

Bevor der Aufbau der Visualisierungen beginnen kann, müssen die Daten fehlerfrei in Logik und in Homogenität in das Tool geladen werden. Zur Umsetzung dieser Anforderungen bietet sich ein ETL Tool an, und mit der Einführung von Tableau Prep Builder im April 2018 gibt der Softwareentwickler dem Anwender ein entsprechendes Tool an die Hand. Die Umsetzung ist sehr gut gelungen und die Bedienung ist sogar Analysten ohne Kenntnisse von Programmiersprachen möglich. Natürlich verfügen die zur Visualisierung gedachten Tools im Produktsortiment (Tableau Desktop, Server und Online) ebenfalls über (gleiche) Werkzeuge zur Datenmanipulierung. Jedoch verfügt Tableau Prep Builder dank seiner erweiterten Visualisierungen zur Transformation und Zusammenführung von Daten über hervorragende Werkzeuge zur Überprüfung und Analyse der Datengrundlage sowie der eigenen Arbeit.

Als Positivbeispiel ist die Visualisierung zu den JOIN-Operationen hervorzuheben, welche dem Anwender auf einen Blick zeigt, wie viele Datensätze vom JOIN betroffen sind und letztendlich auch, wie viele Datensätze in die Output-Tabelle eingeschlossen werden (siehe Grafik).

Zur Datenzusammenführung dienen klassische JOIN- und UNION-Befehle und die Logik entspricht den SQL-Befehlen. Das Ziel dabei ist die Generierung einer Extract-Datei und somit einer zweidimensionalen Tabelle für den Bau von Visualisierungen.

Exkurs – Joins in Power BI:

Erst bei der Visualisierung führt Power BI (im Hintergrund) die Daten durch Joins verschiedener Tabellen zusammen, sofern man vorher ein Datenmodell fehlerfrei definiert hat und die Daten nicht bereits mittels Power Query zusammengeführt hat.

Alternativ können auch diverse Datenquellen in das Visualisierungstool geladen und entsprechend des Power BI-Ansatzes Daten zusammengeführt werden. Dieses sogenannte Data Blending rückt seit der Einführung von Tableau Prep Builder immer mehr in den Hintergrund und Tableau führt die User auch hin zu einer weiteren Komponente: Tableau Prep Conductor. Es ist Bestandteil des bereits erwähnten, kostenpflichtigen Tableau Data Management Add-ons und ergänzt die eingeschränkte Möglichkeit, in Tableau Prep Builder automatisierte Aktualisierungen zu planen.

Kalkulationen können, wie auch bei Power BI, teilweise über ein Userinterface (UI) getätigt werden. Jedoch bietet das UI weniger Möglichkeiten, die wirklich komplizierten Berechnungen vorzunehmen, und der User wird schneller mit der von Tableau entwickelten Sprache konfrontiert. Drei Kategorien von Berechnungen werden unterschieden:

  • Einfache Berechnungen
  • Detailgenauigkeits-Ausdrücke (Level of Detail, LOD)
  • Tabellenberechnungen

Es gibt zwei wesentliche Fragestellungen bei der Auswahl der Berechnungsmethode.

1. Was soll berechnet werden? => Detailgenauigkeit?

Diese Frage klingt auf den ersten Blick simpel, kann aber komplexe Ausmaße annehmen. Tableau gibt hierzu aber einen guten Leitfaden für den Start an die Hand.

2. Wann soll berechnet werden?

Die Wahl der Berechnungsmethode hängt auch davon ab, wann welche Berechnung von der Software durchgeführt wird. Die Reihenfolge der Operationen zeigt die folgende Grafik.

Man braucht einiges an Übung, bis man eine gewisse Selbstsicherheit erlangt hat. Deshalb ist ein strukturiertes Vorgehen für komplexe Vorhaben ratsam.

Daten laden & transformieren: AdventureWorks2017Dataset

Wie bereits im ersten Artikel beschrieben, ist es nicht sehr sinnvoll, ein komplettes Datenmodell in ein BI-Tool zu laden, insbesondere wenn man nur wenige Informationen aus diesem benötigt. Ein für diese Zwecke angepasster View in der Datenbasis wäre aus vielerlei Hinsicht näher an einem Best Practice-Vorgehen. Nicht immer hat man die Möglichkeit, Best Practice im Unternehmen zu leben => siehe Artikel 1 der Serie.

Erst durch die Nutzung von Tableau Prep wurde die komplexe Struktur der Daten deutlich. In Power BI fiel bei der Bereitstellung der Tabellen nicht auf, dass die Adressdaten zu den [Store Contact] nicht in der Tabelle [Adress] zu finden sind. Erst durch die Nutzung von Tableau Prep und einer Analyse zu den Joins, zeigte das Fehlen zuvor genannter Adressen für Stores auf. Weiterhin zeigte die Analyse des Joins von Handelswaren und dazugehöriger Lieferanten auch eine m:n Beziehung auf und somit eine Vervielfachung der Datensätze der output Tabelle.

Kurzum: Tableau Prep ist ein empfehlenswertes Tool, um die Datenbasis schnell zu durchdringen und aufwendige Datenbereitstellungen vorzunehmen.

Daten visualisieren

Erwartungsgemäß sind im Vergleich zwischen Tableau und Power BI einige Visualisierungen leichter und andere dagegen schwerer aufzubauen. Grundsätzlich bieten beide Tools einige vorprogrammierte Visualisierungsobjekte an, welche ohne großen Aufwand erstellt werden können. Interessant wird es beim Vergleich der Detailgenauigkeit der Visualisierungen, wobei es nebensächlich ist, ob es sich dabei um ein Balken- oder Liniendiagramm handelt.

Hands on! Dazu lädt Tableau ein, und das ist auch der beste Weg, um sich mit der Software vertraut zu machen. Für einen einfacheren Start sollte man sich mit zwei wesentlichen Konzepten vertraut machen:

Reihenfolge der Operationen

Yep! Wir hatten das Thema bereits. Ein Blick auf die Grafik beim Basteln einzelner Visualisierungen kann helfen! Jeder Creator und Explorer sollte sich vorher mit der Reihenfolge von Operationen vertraut machen. Das Konzept ist nicht selbsterklärend und Fehler fallen nicht sofort auf. Schaut einmal HIER rein! Tableau hat sich eine Stunde Zeit genommen, um das Konzept anhand von Beispielen zu erklären.

Starre Anordnung von Elementen

Visualisierungen werden erst in einem extra Arbeitsblatt entworfen und können mit anderen Arbeitsblättern in einem Dashboard verbaut werden. Die Anordnung der Elemente auf dem Dashboard kann frei erfolgen und/oder Elemente werden in einer Objekthierarchie abgelegt. Letzteres eignet sich gut für den Bau von Vorlagen und ist somit eine Stärke von Tableau. Das Vorgehen dabei ist nicht trivial, das heißt ein saloppes Reinschmeißen von Visualisierungen führt definitiv nicht zum Ziel.
Tim erklärt ziemlich gut, wie man vorgehen kann => HIER.

Tableau ist aus der Designperspektive limitiert, weshalb das Endergebnis, das Dashboard,  nicht selten sehr eckig und kantig aussieht. Einfache visuelle Anpassungen wie abgerundete Kanten von Arbeitsblättern/Containern sind nicht möglich. Designtechnisch hat Tableau daher noch Luft nach oben!


Der Einstieg für kleine Unternehmen mit Tableau ist nur unter sehr hohem Kostenaufwand möglich, aufgrund von preisintensiven Lizenzen und einer Mindestabnahme an Lizenzen. Aber auch bei einem hohen Bedarf an Lizenzen befindet sich Tableau im höheren Preissegment. Jedoch beinhalten Tableaus Lizenzgebühren bereits Kosten, welche bei der Konkurrenz erst durch die Nutzung ersichtlich werden, da bei ihnen die Höhe der Kosten stärker von der beanspruchten Kapazität abhängig ist. Tableau bietet seinen Kunden damit eine hohe Transparenz über ein zwar preisintensives, aber sehr ausgereiftes Produktportfolio.

Tableau legt mit einer lokalen Option, welche die gleichen Funktionalitäten beinhaltet wie die cloudbasierte Alternative, ein Augenmerk auf Kunden mit strengen Data Governance-Richtlinien. Sandboxed Extensions sind ein weiteres Beispiel für das Bewusstsein für eine hohe Datensicherheit. Jedoch ist das Angebot an Extensions, also das Angebot dritter Entwickler, ausbaufähig. Eine breit aufgestellte Community bietet nicht nur dritten Entwicklern eine gute Geschäftsgrundlage, sondern auch Nutzern zu fast jedem Thema eine Hilfestellung.

Tableau Prep Builder => TOP!

Mit diesem Tool kann die Datengrundlage super einfach analysiert werden und Datenmanipulationen sind einfach durchzuführen. Die Syntax und die Verwendung von Berechnungen bedarf einiger Übung, aber wenn man die wesentlichen Konzepte verstanden hat, dann sind Berechnungen schnell erstellt.

Ein Dashboard kann zu 90 % in fast jedem Tool gleich aussehen. Der Weg dorthin ist oft ein anderer und je nach Anforderung bei einem Tool leichter als bei einem anderen. Tableau bietet ein komplexes Konzept, sodass auch die außergewöhnlichsten Anforderungen erfüllt werden können. Jedoch ist das zugrundliegende Design oft sehr kantig und nicht immer zeitgemäß.

Fortsetzung folgt… MicroStrategy

How Finance Organizations Are Dealing with The Growing Demand for Instant Response Times

The financial industry is one of the most innovative industries that has evolved at an incredibly fast-paced over the past decade. Finance is a complex industry that requires a delicate balance between optimal convenience and security. 

With security being the most important aspect, the role of AI has increased in importance and various financial organizations are taking strides to innovate unique solutions to meet the growing demand for faster and instant response rates. 

In a recent study, it was found that automation and digital intelligence save US banks over $1trillion on an annual basis. From a world perspective, more countries in different parts of the world are adopting AI tools to meet the growing demand for instant response time.

The client experience

Despite the fast rate of digital integration into various industries, clients still want to feel a personal connection to a brand experience. The advances in machine learning have allowed for a vast improvement in personalized services using customer data. This feature uses AI tools to better understand and respond to client needs. 

A feature of this nature allows financial organizations to develop improved products and increase speeds in response rates. The client not only experiences faster service but also gains access to products that are relevant to their needs and interested.

The improved customer experience has also improved by eliminating the need to go to the physical office of a financial institution to solve a problem. The incorporation of chatbots for customer service allows clients to easily solve queries remotely. 

A recent example is the Bank of America’s chatbot, known as Erica, who is accessible at all times of the day is currently used by a million people. This eliminates having to deal with human assistants meaning that it is easier to access solutions. Customer service is on the areas that allow financial institutions to thrive and the client is increasingly demanding optimal customer service. 

Improved security and fraud prevention 

More financial organizations are making use of biometric data to record customer data. Some financial institutions have decided to replace passwords, thus simplifying client verification. Despite the simplicity, it offers a higher level of security beyond a simple pin code. 

In the future, clients are anticipated to simply use their biometrics to access their funds at an ATM or the bank. Another aspect of improving response times to limit cybercrime and prevent fraud by easily identifying client patterns. The knowledge of client patterns allows clients to be contacted in the event of unusual activities. 

Disruption from startup innovation

The term disruption has transformed into a positive term in the past decade because disruptors have created technology that speeds up and streamlines payments, product maintenance for clients and increasing the value chain. 

Financial institutions are finding ways to work collaboratively with disruptors and innovative FinTech companies to create improved technology-driven solutions. The culture of disruption has allowed financial institutions to deliver more innovative money management solutions and simple avenues to process transactions with minimal delays. 

Disruptors generally evolve at a rapid pace and are also becoming institutions that are becoming standalone financial service providers. The expanded competition only creates room for a wide range of institutions to choose from dedicated to solving client problems. 

Using robotics to eliminate the risk

The growing alliance between financial services and technology companies focused on AI allows the financial industry to have a better understanding of consumer patterns to develop products relevant to them. 

The joy of incorporating AI tools means that the client does not have to resort to interacting with a bank teller to solve an issue. The integration of AI tools is a good way to ensure that tasks are performed with minimal human error and eliminate hurdles that arise due to inaccuracies. 

NLP AI Technology has also worked towards assisting financial institutions make informed decisions by developing different useful apps. For example, there are apps that use NLP to gather data on influencers, marketers and blog posts, that data is then used to advise financiers on how to invest. There is also other software that helps digitize financial documentation processes using NLP and that is just a few examples amongst quite a few.

Taking advantage of the sharing economy 

A recent innovation in finance has been the recognition of the power of a shared economy which has been realized in industries such as transport and hospitality. The client is always looking for fast means to meet their needs and the cheapest possible options. 

The rise of digital currencies and the decentralized model have shown banks that people respond to a system that allows for decentralized asset sharing. 

With the rise of cryptocurrency, financial institutions have also started exploring the potential of employing blockchain to create a system that presents a public ledger and improve internal operation within an organization to deliver at high speed. 

Moving infrastructure to the cloud

Financial institutions are growing more and more to use the cloud to manage their operations and this allows for easier management. Financial institutions realize the importance of automating processes such as data management, CRM, accounting and even HR. 

Using analytical tools allows for the fast-tracking of data gathering and delivering solutions to clients. This allows functions like client payment, statement generation, credit checks and more to become automated and more accurate. 

Once again, the issue of cybersecurity is forefronted in machines ‘taking over’ and the concern stems from the fact that the software is being sourced from third parties and requirements in the industry are highly sophisticated. 

The rapid growth of data-driven solutions has placed pressure on financial institutions to work with trustworthy service providers or develop inhouse data management systems to avoid third-party interactions. 


The language of convenience is one that is universal; everyone wants everything to work faster, be delivered to their doorstep and accommodate their needs. The financial industry is no exception to these expectations from customers. Finance organizations are taking the leap into incorporating AI tools to partly manage operations because it simplifies monitoring, reporting and processing large volumes of data. 

The sophistication of analytical tools ensures that issues are resolved before they become larger issues that are beyond an organization’s control. It is certainly exciting to see how financial industries and organizations will transform in 2020 to incorporate tech tools to streamline security and operations.