Business Intelligence Organizations

I am often asked how the Business Intelligence department should be set up and how it should interact and collaborate with other departments. First and foremost: There is no magic recipe here, but every company must find the right organization for itself.

Before we can talk about organization of BI, we need to have a clear definition of roles for team members within a BI department.

A Data Engineer (also Database Developer) uses databases to save structured, semi-structured and unstructured data. He or she is responsible for data cleaning, data availability, data models and also for the database performance. Furthermore, a good Data Engineer has at least basic knowledge about data security and data privacy. A Data Engineer uses SQL and NoSQL-Technologies.

A Data Analyst (also BI Analyst or BI Consultant) uses the data delivered by the Data Engineer to create or adjust data models and implementing business logic in those data models and BI dashboards. He or she needs to understand the needs of the business. This job requires good communication and consulting skills as well as good developing skills in SQL and BI Tools such like MS Power BI, Tableau or Qlik.

A Business Analyst (also Business Data Analyst) is a person form any business department who has basic knowledge in data analysis. He or she has good knowledge in MS Excel and at least basic knowledge in data analysis and BI Tools. A Business Analyst will not create data models in databases but uses existing data models to create dashboards or to adjust existing data analysis applications. Good Business Analyst have SQL Skills.

A Data Scientist is a Data Analyst with extended skills in statistics and machine learning. He or she can use very specific tools and analytical methods for finding pattern in unknow or big data (Data Mining) or to predict events based on pattern calculated by using historized data (Predictive Analytics). Data Scientists work mostly with Python or R programming.

Organization Type 1 – Central Approach (Data Lab)

The first type of organization is the data lab approach. This organization form is easy to manage because it’s focused and therefore clear in terms of budgeting. The data delivery is done centrally by experts and their method and technology knowledge. Consequently, the quality expectation of data delivery and data analysis as well as the whole development process is highest here. Also the data governance is simple and the responsibilities clearly adjustable. Not to be underestimated is the aspect of recruiting, because new employees and qualified applicants like to join a central team of experts.

However, this form of organization requires that the company has the right working attitude, especially in the business intelligence department. A centralized business intelligence department acts as a shared service. Accordingly, customer-oriented thinking becomes a prerequisite for the company’s success – and customers here are the other departments that need access to the capacities of those centralized data experts. Communication boundaries must be overcome and ways of simple and effective communication must be found.

Organization Type 2 – Stakeholder Focus Approach

Other companies want to shift more responsibility for data governance, and especially data use and analytics, to those departments where data plays a key role right now. A central business intelligence department manages its own projects, which have a meaning for the entire company. The specialist departments, which have a special need for data analysis, have their own data experts who carry out critical projects for the specialist department. The central Business Intelligence department does not only provide the technical delivery of data, but also through methodical consulting. Although most of the responsibility lies with the Business Intelligence department, some other data-focused departments are at least co-responsible.

The advantage is obvious: There are special data experts who work deeper in the actual departments and feel more connected and responsible to them. The technical-business focus lies on pain points of the company.

However, this form of Ogranization also has decisive disadvantages: The danger of developing isolated solutions that are so special in some specific areas that they will not really work company-wide increases. Typically the company has to deal with asymmetrical growth of data analytics
know-how. Managing data governance is more complex and recruitment is becoming more difficult as the business intelligence department is weakened and smaller, and data professionals for other departments need to have more business focus, which means they are looking for more specialized profiles.

Organization Type 3 – Decentral Approach

Some companies are also taking a more extreme approach in the other direction. The Business Intelligence department now has only Data Engineers building and maintaining the data warehouse or data lake. As a result, the central department only provides data; it is used and analyzed in all other departments, specifically for the respective applications.

The advantage lies in the personal responsibility of the respective departments as „pain points“ of the company are in focus in belief that business departments know their problems and solutions better than any other department does. Highly specialized data experts can understand colleagues of their own department well and there is no no shared service mindset neccessary, except for the data delivery.

Of course, this organizational form has clear disadvantages since many isolated solutions are unavoidable and the development process of each data-driven solution will be inefficient. These insular solutions may work with luck for your own department, but not for the whole company. There is no one single source of truth. The recruiting process is more difficult as it requires more specialized data experts with more business background. We have to expect an asymmetrical growth of data analytics know-how and a difficult data governance.

 

Veröffentlichung der Data Leader Days 2018 Vorträge auf YouTube

Für alle Interessenten, die es im vergangenen Jahr leider nicht geschafft haben, persönlich an den Data Leader Days 2018 teilzunehmen, gibt es direkt zu Beginn des Jahres 2019 erfreuliche Nachrichten: die Vorträge sind ab heute als Videoaufnahmen auf unserem Youtube-Kanal verfügbar:

 

Besuchen Sie unseren Youtube-Kanal, um sich alle Video Beiträge anzusehen:

https://www.youtube.com/channel/UCXRUMGr6MZPMC5WAKiBLdVg/videos

Die Data Leader Days freuen sich auch im nächsten Jahr wieder auf zahlreiche Teilnehmer. Das Event findet am 13. und 14. November 2019 wieder in Berlin statt.

Data Leader Days 2018 – Review

Das Who’s Who der Datenwirtschaft auf den Data Leader Days 2018

Berlin, Dezember 2018: Die Data Leader Days am 14./15. November 2018 im Berliner Spreespeicher haben erneut die Entscheider aus der Business- und Digitalwelt versammelt und wichtige Impulse ausgesendet. Die in diesem Jahr zum dritten Mal stattfindende Veranstaltung verzeichnete mit knapp 300 Teilnehmern einen enormen Besucherzuwachs. Organisiert wurde die Konferenz von DATANOMIQ und dem Connected Industry.

Der Auftakttag stand ganz unter dem Zeichen von Commercial und Finance Data: Besondere Highlights waren hier die Vorträge von Dr. Joachim Schmalzl, Vorstandsmitglied des Dt. Sparkassen- und Giroverbands, der auf die Fortschritte der Sparkassen bei der Umsetzung von digitalen Innovationen einging sowie Marcus Hartmann, Chief Data Officer der ProSieben Sat. 1 Media mit seiner Keynote. Im Fokus des zweiten Tages standen Industrial und Automotive Data. Hier konnten Digitalmanager von BASF, Heidelberger Druckmaschinen, E.ON, Wittenstein, Vodafone, Schaeffler und Airbus anhand von Live Demos und Use Cases die Themen Data Science & Machine Learning, Data Engineering sowie Data Visualization vorstellen.

Die Data Leader Days freuen sich auch im nächsten Jahr wieder auf eine große Resonanz. Das Event findet wieder in Berlin am 13./14. November 2019 statt.

Data Leader Days Sponsors and Audience

Location at Data Hacker Days announced

We are happy to announce your our venue for the Data Hacker Days: Kühlhaus Berlin.

Data Hacker Days will take place from 28th to 30st May 2019 in Berlin.

The industrial look of Kühlhaus Berlin is a perfect place to challenge hackers to work and solve the hackathon. The backyard will be a chill area and to take a break or networking with other people.  

More information about Data Hacker you can find on the website: https://datahackerdays.com/

Like our facebook website: https://www.facebook.com/Data-Hacker-Days-278017336397423/

Freier Eintritt für Young Professionals zu den Data Leader Days 2018

Jetzt bewerben und kostenfrei beim Spitzenevent der Datenwirtschaft am 14. oder 15. November in Berlin dabei sein!
Die Data Leader Days senden regelmäßig wichtige Impulse in die Big Data und KI-Welt aus und sind ein führendes Forum für Wissens-, Ideen- und Informationsaustausch. Die Spitzen von Anwenderunternehmen zeigen exklusiv in einem innovativen Programm mit Keynote, Präsentationen sowie Use & Business Cases auf, wie Digitalisierung und Künstliche Intelligenz umgesetzt und zum neuen Wettbewerbsvorteil werden.

Zu den Speakern gehören die Data Leader von E.ON, Pro7Sat1, Deutscher Sparkassen- und Giroverband, Airbus, Wittenstein, BASF, Merck, Heidelberger Druckmaschinen, Vodafone, FTI und von weiteren Unternehmen.

Bewerbe Dich bis zum 02.11.2018 mit einem kurzen Statement, warum Du dabei sein möchtest! Schicke mir Dein Statement an linhchi.nguyen@datanomiq.de und überzeuge uns.
Ist dein Statement aussagekräftig und überzeugend, laden wir Dich kostenlos zu einem der beiden Veranstaltungstage ein.

My Desk for Data Science

In my last post I anounced a blog parade about what a data scientist’s workplace might look like.

Here are some photos of my desk and my answers to the questions:

How many monitors do you use (or wish to have)?

I am mostly working at my desk in my office with a tower PC and three monitors.
I definitely need at least three monitors to work productively as a data scientist. Who does not know this: On the left monitor the data model is displayed, on the right monitor the data mapping and in the middle I do my work: programming the analysis scripts.

What hardware do you use? Apple? Dell? Lenovo? Others?

I am note an Apple guy. When I need to work mobile, I like to use ThinkPad notebooks. The ThinkPads are (in my experience) very robust and are therefore particularly good for mobile work. Besides, those notebooks look conservative and so I’m not sad if there comes a scratch on the notebook. However, I do not solve particularly challenging analysis tasks on a notebook, because I need my monitors for that.

Which OS do you use (or prefer)? MacOS, Linux, Windows? Virtual Machines?

As a data scientist, I have to be able to communicate well with my clients and they usually use Microsoft Windows as their operating system. I also use Windows as my main operating system. Of course, all our servers run on Linux Debian, but most of my tasks are done directly on Windows.
For some notebooks, I have set up a dual boot, because sometimes I need to start native Linux, for all other cases I work with virtual machines (Linux Ubuntu or Linux Mint).

What are your favorite databases, programming languages and tools?

I prefer the Microsoft SQL Server (T-SQL), C# and Python (pandas, numpy, scikit-learn). This is my world. But my customers are kings, therefore I am working with Postgre SQL, MongoDB, Neo4J, Tableau, Qlik Sense, Celonis and a lot more. I like to get used to new tools and technologies again and again. This is one of the benefits of being a data scientist.

Which data dou you analyze on your local hardware? Which in server clusters or clouds?

There have been few cases yet, where I analyzed really big data. In cases of analyzing big data we use horizontally scalable systems like Hadoop and Spark. But we also have customers analyzing middle-sized data (more than 10 TB but less than 100 TB) on one big server which is vertically scalable. Most of my customers just want to gather data to answer questions on not so big amounts of data. Everything less than 10TB we can do on a highend workstation.

If you use clouds, do you prefer Azure, AWS, Google oder others?

Microsoft Azure! I am used to tools provided by Microsoft and I think Azure is a well preconfigured cloud solution.

Where do you make your notes/memos/sketches. On paper or digital?

My calender is managed digital, because I just need to know everywhere what appointments I have. But my I prefer to wirte down my thoughts on paper and that´s why I have several paper-notebooks.

Now it is your turn: Join our Blog Parade!

So what does your workplace look like? Show your desk on your blog until 31/12/2017 and we will show a short introduction of your post here on the Data Science Blog!

 

Success Criteria Process Mining

Process Mining is much more than the automatic drawing of process models.

Process mining is on the rise. By using Process mining, organizations can see how their processes really operate [1]. The results are amazing new insights about these processes that cannot be obtained in any other way. However, there are a few things that can go wrong. In this article, Frank van Geffen and Anne Rozinat give you tips about the pitfalls and advice that will help you to make your first process mining project as successful as it can be. Read more