Benjamin Aunkofer

Datenanalytische Denkweise: Müssen Führungskräfte Data Science verstehen?

/in , , , , /by

Die Digitalisierung ist in Deutschland bereits seit Jahrzehnten am Voranschreiten. Im Gegensatz zum verbreiteten Glauben, dass die Digitalisierung erst mit der Innovation der Smartphones ihren Anfang fand, war der erste Schritt bereits die Einführung von ERP-Systemen. Sicherlich gibt es hier noch einiges zu tun, jedoch hat die Digitalisierung meines Erachtens nach das Plateau der Produktivität schon bald erreicht – Ganz im Gegensatz zur Datennutzung!

Die Digitalisierung erzeugt eine exponentiell anwachsende Menge an Daten, die ein hohes Potenzial an neuen Erkenntnissen für Medizin, Biologie, Agrawirtschaft, Verkehrswesen und die Geschäftswelt bedeuten. Es mag hier und da an Fachexperten fehlen, die wissen, wie mit großen und heterogenen Daten zu hantieren ist und wie sie zu analysieren sind. Das Aufleben dieser Experenberufe und auch neue Studengänge sorgen jedoch dafür, dass dem Mangel ein gewisser Nachwuchs entgegen steht.

Doch wie sieht es mit Führungskräften aus? Müssen Entscheider verstehen, was ein Data Engineer oder ein Data Scientist tut, wie seine Methoden funktionieren und an welche Grenzen eingesetzte Software stößt?

Datenanalytische Denkweise ist ein strategisches Gut

Als Führungskraft müssen Sie unternehmerisch denken und handeln. Wenn Sie eine neue geschäftliche Herausforderung erfolgreich bewältigen möchten, müssen Sie selbst Ideen entwickeln – oder diese zumindest bewerten – können, wie in Daten Antworten für eine Lösung gefunden werden können. Die meisten Führungskräfte reden sich erfahrungsgemäß damit heraus, dass sie selbst keine höheren Datenanalysen durchführen müssen. Unternehmen werden gegenwärtig bereits von Datenanalysten vorangetrieben und für die nahe Zukunft besteht kein Zweifel an der zunehmenden Bedeutung von Datenexperten für die Entscheidungsfindung nicht nur auf der operativen Ebene, bei der Dateningenieure sehr viele Entscheidungen automatisieren werden, sondern auch auf der strategischen Ebene.

Sie müssen kein Data Scientist sein, aber Grundkenntnisse sind der Schlüssel zum Erfolg

Hinter den Begriffen Big Data und Advanced Analytics – teilweise verhasste Buzzwords – stecken reale Methoden und Technologien, die eine Führungskraft richtig einordnen können muss, um über Projekte und Invesitionen entscheiden zu können. Zumindest müssen Manager ihre Mitarbeiter kennen und deren Rollen und Fähigkeiten verstehen, dabei dürfen sie sich keinesfalls auf andere verlassen. Übrigens wissen auch viele Recruiter nicht, wen genau sie eigentlich suchen!

Der Weg zum Data-Driven Decision Making: Abgrenzung von IT-Administration, Data Engineering und Data Science, in Anlehnung an Data Science for Business: What you need to know about data mining and data-analytic thinking

Stark vereinfacht betrachtet, dreht sich dabei alles um Analysemethodik, Datenbanken und Programmiersprachen. Selbst unabhängig vom aktuellen Analytcs-Trend, fördert eine Einarbeitung in diese Themenfelder das logische denken und kann auch sehr viel Spaß machen. Als positiven Nebeneffekt werden Sie eine noch unternehmerischere und kreativere Denkweise entwickeln!

Datenaffinität ist ein Karriere-Turbo!

Nicht nur der Bedarf an Fachexperten für Data Science und Data Engineering steigt, sondern auch der Bedarf an Führungskräften bzw. Manager. Sicherlich ist der Bedarf an Führungskräften quantitativ stets geringer als der für Fachexperten, immerhin braucht jedes Team nur eine Führung, jedoch wird hier oft vergessen, dass insbesondere Data Science kein Selbstzweck ist, sondern für alle Fachbereiche (mit unterschiedlicher Priorisierung) Dienste leisten kann. Daten-Projekte scheitern entweder am Fehlen der datenaffinen Fachkräfte oder am Fehlen von datenaffinen Führungskräften in den Fachabteilungen. Unverständnisvolle Fachbereiche tendieren schnell zur Verweigerung der Mitwirkung – bis hin zur klaren Arbeitsverweigerung – auf Grund fehlender Expertise bei Führungspersonen.

Andersrum betrachtet, werden Sie als Führungskraft Ihren Marktwert deutlich steigern, wenn Sie ein oder zwei erfolgreiche Projekte in Ihr Portfolio aufnehmen können, die im engen Bezug zur Datennutzung stehen.

Mit einem Data Science Team: Immer einen Schritt voraus!

Führungskräfte, die zukünftige Herausforderungen meistern möchten, müssen selbst zwar nicht Data Scientist werden, jedoch dazu in der Lage sein, ein kleines Data Science Team führen zu können. Möglicherweise handelt es sich dabei nicht direkt um Ihr Team, vielleicht ist es jedoch Ihre Aufgabe, das Team durch Ihren Fachbereich zu leiten. Data Science Teams können zwar auch direkt in einer Fachabteilung angesiedelt sein, sind häufig jedoch zentrale Stabstellen.

Müssen Sie ein solches Team für Ihren Fachbereich begleiten, ist es selbstverständlich notwendig, dass sie sich über gängige Verfahren der Datenanalyse, also auch der Statistik, und der maschinellen Lernverfahren ein genaueres Bild machen. Erkennen Data Scientists, dass Sie sich als Führungskraft mit den Verfahren auseinander gesetzt haben, die wichtigsten Prozeduren, deren Anforderungen und potenziellen Ergebnisse kennen oder einschätzen können, werden Sie mit entsprechendem Respekt belohnt und Ihre Data Scientists werden Ihnen gute Berater sein, wie sie Ihre unternehmerischen Ziele mit Daten erreichen werden.

Buchempfehlung:

Data Science für Unternehmen: Data Mining und datenanalytisches Denken praktisch anwenden (mitp Business)

Lesetipps:

Benjamin Aunkofer

My Desk for Data Science

/in , , , , /by

In my last post I anounced a blog parade about what a data scientist’s workplace might look like.

Here are some photos of my desk and my answers to the questions:

How many monitors do you use (or wish to have)?

I am mostly working at my desk in my office with a tower PC and three monitors.
I definitely need at least three monitors to work productively as a data scientist. Who does not know this: On the left monitor the data model is displayed, on the right monitor the data mapping and in the middle I do my work: programming the analysis scripts.

What hardware do you use? Apple? Dell? Lenovo? Others?

I am note an Apple guy. When I need to work mobile, I like to use ThinkPad notebooks. The ThinkPads are (in my experience) very robust and are therefore particularly good for mobile work. Besides, those notebooks look conservative and so I’m not sad if there comes a scratch on the notebook. However, I do not solve particularly challenging analysis tasks on a notebook, because I need my monitors for that.

Which OS do you use (or prefer)? MacOS, Linux, Windows? Virtual Machines?

As a data scientist, I have to be able to communicate well with my clients and they usually use Microsoft Windows as their operating system. I also use Windows as my main operating system. Of course, all our servers run on Linux Debian, but most of my tasks are done directly on Windows.
For some notebooks, I have set up a dual boot, because sometimes I need to start native Linux, for all other cases I work with virtual machines (Linux Ubuntu or Linux Mint).

What are your favorite databases, programming languages and tools?

I prefer the Microsoft SQL Server (T-SQL), C# and Python (pandas, numpy, scikit-learn). This is my world. But my customers are kings, therefore I am working with Postgre SQL, MongoDB, Neo4J, Tableau, Qlik Sense, Celonis and a lot more. I like to get used to new tools and technologies again and again. This is one of the benefits of being a data scientist.

Which data dou you analyze on your local hardware? Which in server clusters or clouds?

There have been few cases yet, where I analyzed really big data. In cases of analyzing big data we use horizontally scalable systems like Hadoop and Spark. But we also have customers analyzing middle-sized data (more than 10 TB but less than 100 TB) on one big server which is vertically scalable. Most of my customers just want to gather data to answer questions on not so big amounts of data. Everything less than 10TB we can do on a highend workstation.

If you use clouds, do you prefer Azure, AWS, Google oder others?

Microsoft Azure! I am used to tools provided by Microsoft and I think Azure is a well preconfigured cloud solution.

Where do you make your notes/memos/sketches. On paper or digital?

My calender is managed digital, because I just need to know everywhere what appointments I have. But my I prefer to wirte down my thoughts on paper and that´s why I have several paper-notebooks.

Now it is your turn: Join our Blog Parade!

So what does your workplace look like? Show your desk on your blog until 31/12/2017 and we will show a short introduction of your post here on the Data Science Blog!

 

Benjamin Aunkofer

Show your Data Science Workplace!

/in , , , /by

The job of a data scientist is often a mystery to outsiders. Of course, you do not really need much more than a medium-sized notebook to use data science methods for finding value in data. Nevertheless, data science workplaces can look so different and, let’s say, interesting. And that’s why I want to launch a blog parade – which I want to start with this article – where you as a Data Scientist or Data Engineer can show your workplace and explain what tools a data scientist in your opinion really needs.

I am very curious how many monitors you prefer, whether you use Apple, Dell, HP or Lenovo, MacOS, Linux or Windows, etc., etc. And of course, do you like a clean or messy desk?

What is a Blog Parade?

A blog parade is a call to blog owners to report on a specific topic. Everyone who participates in the blog parade, write on their blog a contribution to the topic. The organizer of the blog parade collects all the articles and will recap those articles in a short form together, of course with links to the articles.

How can I participate?

Write an article on your blog! Mention this blog parade here, show and explain your workplace (your desk with your technical equipment) in an article. If you’re missing your own blog, articles can also be posted directly to LinkedIn (LinkedIn has its own blogging feature that every LinkedIn member can use). Alternative – as a last resort – it would also be possible to send me your article with a photo about your workplace directly to: redaktion@data-science-blog.com.
Please make me aware of an article, via e-mail or with a comment (below) on this article.

Who can participate?

Any data scientist or anyone close to Data Science: Everyone concerned with topics such as data analytics, data engineering or data security. Please do not over-define data science here, but keep it in a nutshell, so that all professionals who manage and analyze data can join in with a clear conscience.

And yes, I will participate too. I will propably be the first who write an article about my workplace (I just need a new photo of my desk).

When does the article have to be finished?

By 31/12/2017, the article must have been published on your blog (or LinkedIn or wherever) and the release has to be reported to me.
But beware: Anyone who has previously written an article will also be linked earlier. After all, reporting on your article will take place immediately after I hear about it.
If you publish an artcile tomorrow, it will be shown the day after tomorrow here on the Data Science Blog.

What is in it for me to join?

Nothing! Except perhaps the fun factor of sharing your idea of ​​a nice desk for a data expert with others, so as to share creativity or a certain belief in what a data scientist needs.
Well and for bloggers: There is a great backlink from this data science blog for you 🙂

What should I write? What are the minimum requirements of content?

The article does not have to (but may be) particularly long. Anyway, here on this data science blog only a shortened version of your article will appear (with a link, of course).

Minimum requirments:

  • Show a photo (at least one!) of your workplace desk!
  • And tell us something about:
    • How many monitors do you use (or wish to have)?
    • What hardware do you use? Apple? Dell? Lenovo? Others?
    • Which OS do you use (or prefer)? MacOS, Linux, Windows? Virtual Machines?
    • What are your favorite databases, programming languages and tools? (e.g. Python, R, SAS, Postgre, Neo4J,…)
    • Which data dou you analyze on your local hardware? Which in server clusters or clouds?
    • If you use clouds, do you prefer Azure, AWS, Google oder others?
    • Where do you make your notes/memos/sketches. On paper or digital?

Not allowed:
Of course, please do not provide any information, which could endanger your company`s IT security.

Absolutly allowed:
Bringing some joke into the matter 🙂 We are happy to vote in the comments on the best or funniest desk for election, there may be also a winner later!

The resulting Blog Posts: https://data-science-blog.com/data-science-insights/show-your-desk/

 

Thomas Blanchard

The importance of domain knowledge – A healthcare data science perspective

/in , , , , , , , /by

Data scientists have (and need) many skills. They are frequently either former academic researchers or software engineers, with knowledge and skills in statistics, programming, machine learning, and many other domains of mathematics and computer science. These skills are general and allow data scientists to offer valuable services to almost any field. However, data scientists in some cases find themselves in industries they have relatively little knowledge of.

This is especially true in the healthcare field. In healthcare, there is an enormous amount of important clinical knowledge that might be relevant to a data scientist. It is unreasonable to expect a data scientist to not only have all of the skills typically required of a data scientist, but to also have all of the knowledge a medical professional may have.

Why is domain knowledge necessary?

This lack of domain knowledge, while perfectly understandable, can be a major barrier to healthcare data scientists. For one thing, it’s difficult to come up with project ideas in a domain that you don’t know much about. It can also be difficult to determine the type of data that may be helpful for a project – if you want to build a model to predict a health outcome (for example, whether a patient has or is likely to develop a gastrointestinal bleed), you need to know what types of variables might be related to this outcome so you can make sure to gather the right data.

Knowing the domain is useful not only for figuring out projects and how to approach them, but also for having rules of thumb for sanity checks on the data. Knowing how data is captured (is it hand-entered? Is it from machines that can give false readings for any number of reasons?) can help a data scientist with data cleaning and from going too far down the wrong path. It can also inform what true outliers are and which values might just be due to measurement error.

Often the most challenging part of building a machine learning model is feature engineering. Understanding clinical variables and how they relate to a health outcome is extremely important for this. Is a long history of high blood pressure important for predicting heart problems, or is only very recent history? How long a time horizon is considered ‘long’ or ‘short’ in this context? What other variables might be related to this health outcome? Knowing the domain can help direct the data exploration and greatly speed (and enhance) the feature engineering process.

Once features are generated, knowing what relationships between variables are plausible helps for basic sanity checks. If you’re finding the best predictor of hospitalization is the patient’s eye color, this might indicate an issue with your code. Being able to glance at the outcome of a model and determine if they make sense goes a long way for quality assurance of any analytical work.

Finally, one of the biggest reasons a strong understanding of the data is important is because you have to interpret the results of analyses and modeling work. Knowing what results are important and which are trivial is important for the presentation and communication of results. An analysis that determines there is a strong relationship between age and mortality is probably well-known to clinicians, while weaker but more surprising associations may be of more use. It’s also important to know what results are actionable. An analysis that finds that patients who are elderly are likely to end up hospitalized is less useful for trying to determine the best way to reduce hospitalizations (at least, without further context).

How do you get domain knowledge?

In some industries, such as tech, it’s fairly easy and straightforward to see an end-user’s prospective. By simply viewing a website or piece of software from the user’s point of view, a data scientist can gain a lot of the needed context and background knowledge needed to understand where their data is coming from and how their model output is being used. In the healthcare industry, it’s more difficult. A data scientist can’t easily choose to go through med school or the experience of being treated for a chronic illness. This means there is no easy single answer to where to gain domain knowledge. However, there are many avenues available.

Reading literature and attending presentations can boost one’s domain knowledge. However, it’s often difficult to find resources that are penetrable for someone who is not already a clinician. To gain deep knowledge, one needs to be steeped in the topic. One important avenue to doing this is through the establishment of good relationships with clinicians. Clinicians can be powerful allies that can help point you in the right direction for understanding your data, and simply by chatting with them you can gain important insights. They can also help you visit the clinics or practices to interact with the people that perform the procedures or even watch the procedures being done. At Fresenius Medical Care, where I work, members of my team regularly visit clinics. I have in the last year visited one of our dialysis clinics, a nephrology practice, and a vascular care unit. These experiences have been invaluable to me in developing my knowledge of the treatment of chronic illnesses.

In conclusion, it is crucial for data scientists to acquire basic familiarity in the field they are working in and in being part of collaborative teams that include people who are technically knowledgeable in the field they work in. This said, acquiring even an essential understanding (such as “Medicine 101”) may go a long way for the data scientists in being able to become self-sufficient in essential feature selection and design.

 

Benjamin Aunkofer

Data Science vs Data Engineering

/in , , , , , , /by

The job of the Data Scientist is actually a fairly new trend, and yet other job titles are coming to us. “Is this really necessary?”, Some will ask. But the answer is clear: yes!

There are situations, every Data Scientist know: a recruiter calls, speaks about a great new challenge for a Data Scientist as you obviously claim on your LinkedIn profile, but in the discussion of the vacancy it quickly becomes clear that you have almost none of the required skills. This mismatch is mainly due to the fact that under the job of the Data Scientist all possible activity profiles, method and tool knowledge are summarized, which a single person can hardly learn in his life. Many open jobs, which are to be called under the name Data Science, describe rather the professional image of the Data Engineer.

Read this article in German:
“Data Science vs Data Engineering – Wo liegen die Unterschiede?“

What is a Data Engineer?

Data engineering is primarily about collecting or generating data, storing, historicalizing, processing, adapting and submitting data to subsequent instances. A Data Engineer, often also named as Big Data Engineer or Big Data Architect, models scalable database and data flow architectures, develops and improves the IT infrastructure on the hardware and software side, deals with topics such as IT Security , Data Security and Data Protection. A Data Engineer is, as required, a partial administrator of the IT systems and also a software developer, since he or she extends the software landscape with his own components. In addition to the tasks in the field of ETL / Data Warehousing, he also carries out analyzes, for example, to investigate data quality or user access. A Data Engineer mainly works with databases and data warehousing tools.

A Data Engineer is talented as an educated engineer or computer scientist and rather far away from the actual core business of the company. The Data Engineer’s career stages are usually something like:

  1. (Big) Data Architect
  2. BI Architect
  3. Senior Data Engineer
  4. Data Engineer

What makes a Data Scientist?

Although there may be many intersections with the Data Engineer’s field of activity, the Data Scientist can be distinguished by using his working time as much as possible to analyze the available data in an exploratory and targeted manner, to visualize the analysis results and to convert them into a red thread (storytelling). Unlike the Data Engineer, a data scientist rarely sees into a data center, because he picks up data via interfaces provided by the Data Engineer or provides by other resources.

A Data Scientist deals with mathematical models, works mainly with statistical procedures, and applies them to the data to generate knowledge. Common methods of Data Mining, Machine Learning and Predictive Modeling should be known to a Data Scientist. Data Scientists basically work close to the department and need appropriate expertise. Data Scientists use proprietary tools (e.g. Tools by IBM, SAS or Qlik) and program their own analyzes, for example, in Scala, Java, Python, Julia, or R. Using such programming languages and data science libraries (e.g. Mahout, MLlib, Scikit-Learn or TensorFlow) is often considered as advanced data science.

Data Scientists can have diverse academic backgrounds, some are computer scientists or engineers for electrical engineering, others are physicists or mathematicians, not a few have economical backgrounds. Common career levels could be:

  1. Chief Data Scientist
  2. Senior Data Scientist
  3. Data Scientist
  4. Data Analyst oder Junior Data Scientist

Data Scientist vs Data Analyst

I am often asked what the difference between a Data Scientist and a Data Analyst would be, or whether there would be a distinction criterion at all:

In my experience, the term Data Scientist stands for the new challenges for the classical concept of Data Analysts. A Data Analyst performs data analysis like a Data Scientist. More complex topics such as predictive analytics, machine learning or artificial intelligence are topics for a Data Scientist. In other words, a Data Scientist is a Data Analyst++ (one step above the Data Analyst).

And how about being a Business Analyst?

Business Analysts can (but need not) be Data Analysts. In any case, they have a very strong relationship with the core business of the company. Business Analytics is about analyzing business models and business successes. The analysis of business success is usually carried out by IT, and many business analysts are starting a career as Data Analyst now. Dashboards, KPIs and SQL are the tools of a good business analyst, but there might be a lot business analysts, who are just analysing business models by reading the newspaper…

Redaktion

Weiterbildungsangebote zu Data Science und R an der TU Dortmund

/in , , , , , , /by

Anzeige: Interessante Weiterbildungsangebote zu Data Science und Programmiersprache R an der TU Dortmund

Das Zertifikatsstudium „Data Science and Big Data“ an der Technischen Universität Dortmund startet im Januar 2018 in den zweiten Durchgang. Aufbauend auf datenwissenschaftlichen Erkenntnissen steht die praxisnahe Umsetzung eines eigenen Big-Data Projekts im Fokus der Weiterbildung. Mithilfe von Methoden aus den Disziplinen Statistik, Informatik und Journalistik erwerben die Teilnehmerinnen und Teilnehmer wertvolle Kompetenzen in den Bereichen Datenanalyse, Datenmanagement und Ergebnisdarstellung. Die Bewerbungsphase läuft noch bis zum 8. November 2017. Mehr Infos finden Sie unter: https://data-science-blog.com/tu-dortmund-berufsbegleitendes-zertifikatsstudium/

Ganz neu ist ein weiteres Tagesseminarangebot im Bereich Data Science ab Frühjahr 2018: Dortmunder R-Kurse. Hier vermitteln Experten in Kursen für Anfänger und Fortgeschrittene die praktische Anwendung der Statistiksoftware R. Näheres dazu gibt es hier: www.zhb.tu-dortmund.de/r-kurse

 

Benjamin Aunkofer

Data Science Knowledge Stack – Abstraction of the Data Science Skillset

/in , , , , , , , , /by

What must a Data Scientist be able to do? Which skills does as Data Scientist need to have? This question has often been asked and frequently answered by several Data Science Experts. In fact, it is now quite clear what kind of problems a Data Scientist should be able to solve and which skills are necessary for that. I would like to try to bring this consensus into a visual graph: a layer model, similar to the OSI layer model (which any data scientist should know too, by the way).
I’m giving introductory seminars in Data Science for merchants and engineers and in those seminars I always start explaining what we need to work out together in theory and practice-oriented exercises. Against this background, I came up with the idea for this layer model. Because with my seminars the problem already starts: I am giving seminars for Data Science for Business Analytics with Python. So not for medical analyzes and not with R or Julia. So I do not give a general knowledge of Data Science, but a very specific direction.

A Data Scientist must deal with problems at different levels in any Data Science project, for example, the data access does not work as planned or the data has a different structure than expected. A Data Scientist can spend hours debating its own source code or learning the ropes of new DataScience packages for its chosen programming language. Also, the right algorithms for data evaluation must be selected, properly parameterized and tested, sometimes it turns out that the selected methods were not the optimal ones. Ultimately, we are not doing Data Science all day for fun, but for generating value for a department and a data scientist is also faced with special challenges at this level, at least a basic knowledge of the expertise of that department is a must have.

Read this article in German:
“Data Science Knowledge Stack – Was ein Data Scientist können muss“

Data Science Knowledge Stack

With the Data Science Knowledge Stack, I would like to provide a structured insight into the tasks and challenges a Data Scientist has to face. The layers of the stack also represent a bidirectional flow from top to bottom and from bottom to top, because Data Science as a discipline is also bidirectional: we try to answer questions with data, or we look at the potentials in the data to answer previously unsolicited questions.

The DataScience Knowledge Stack consists of six layers:

Database Technology Knowledge

A Data Scientist works with data which is rarely directly structured in a CSV file, but usually in one or more databases that are subject to their own rules. In particular, business data, for example from the ERP or CRM system, are available in relational databases, often from Microsoft, Oracle, SAP or an open source alternative. A good Data Scientist is not only familiar with Structured Query Language (SQL), but is also aware of the importance of relational linked data models, so he also knows the principle of data table normalization.

Other types of databases, so-called NoSQL databases (Not only SQL) are based on file formats, column or graph orientation, such as MongoDB, Cassandra or GraphDB. Some of these databases use their own programming languages ​​(for example JavaScript at MongoDB or the graph-oriented database Neo4J has its own language called Cypher). Some of these databases provide alternative access via SQL (such as Hive for Hadoop).

A data scientist has to cope with different database systems and has to master at least SQL – the quasi-standard for data processing.

Data Access & Transformation Knowledge

If data are given in a database, Data Scientists can perform simple (and not so simple) analyzes directly on the database. But how do we get the data into our special analysis tools? To do this, a Data Scientist must know how to export data from the database. For one-time actions, an export can be a CSV file, but which separators and text qualifiers should be used? Possibly, the export is too large, so the file must be split.
If there is a direct and synchronous data connection between the analysis tool and the database, interfaces like REST, ODBC or JDBC come into play. Sometimes a socket connection must also be established and the principle of a client-server architecture should be known. Synchronous and asynchronous encryption methods should also be familiar to a Data Scientist, as confidential data are often used, and a minimum level of security is most important for business applications.

Many datasets are not structured in a database but are so-called unstructured or semi-structured data from documents or from Internet sources. And again we have interfaces, a frequent entry point for Data Scientists is, for example, the Twitter API. Sometimes we want to stream data in near real-time, let it be machine data or social media messages. This can be quite demanding, so the data streaming is almost a discipline with which a Data Scientist can come into contact quickly.

Programming Language Knowledge

Programming languages ​​are tools for Data Scientists to process data and automate processing. Data Scientists are usually no real software developers and they do not have to worry about software security or economy. However, a certain basic knowledge about software architectures often helps because some Data Science programs can be going to be integrated into an IT landscape of the company. The understanding of object-oriented programming and the good knowledge of the syntax of the selected programming languages ​​are essential, especially since not every programming language is the most useful for all projects.

At the level of the programming language, there is already a lot of snares in the programming language that are based on the programming language itself, as each has its own faults and details determine whether an analysis is done correctly or incorrectly: for example, whether data objects are copied or linked as reference, or how NULL/NaN values ​​are treated.

Data Science Tool & Library Knowledge

Once a data scientist has loaded the data into his favorite tool, for example, one of IBM, SAS or an open source alternative such as Octave, the core work just began. However, these tools are not self-explanatory and therefore there is a wide range of certification options for various Data Science tools. Many (if not most) Data Scientists work mostly directly with a programming language, but this alone is not enough to effectively perform statistical data analysis or machine learning: We use Data Science libraries (packages) that provide data structures and methods as a groundwork and thus extend the programming language to a real Data Science toolset. Such a library, for example Scikit-Learn for Python, is a collection of methods implemented in the programming language. The use of such libraries, however, is intended to be learned and therefore requires familiarization and practical experience for reliable application.

When it comes to Big Data Analytics, the analysis of particularly large data, we enter the field of Distributed Computing. Tools (frameworks) such as Apache Hadoop, Apache Spark or Apache Flink allows us to process and analyze data in parallel on multiple servers. These tools also provide their own libraries for machine learning, such as Mahout, MLlib and FlinkML.

Data Science Method Knowledge

A Data Scientist is not simply an operator of tools, he uses the tools to apply his analysis methods to data he has selected for to reach the project targets. These analysis methods are, for example, descriptive statistics, estimation methods or hypothesis tests. Somewhat more mathematical are methods of machine learning for data mining, such as clustering or dimensional reduction, or more toward automated decision making through classification or regression.

Machine learning methods generally do not work immediately, they have to be improved using optimization methods like the gradient method. A Data Scientist must be able to detect under- and overfitting, and he must prove that the prediction results for the planned deployment are accurate enough.

Special applications require special knowledge, which applies, for example, to the fields of image recognition (Visual Computing) or the processing of human language (Natural Language Processiong). At this point, we open the door to deep learning.

Expertise

Data Science is not an end in itself, but a discipline that would like to answer questions from other expertise fields with data. For this reason, Data Science is very diverse. Business economists need data scientists to analyze financial transactions, for example, to identify fraud scenarios or to better understand customer needs, or to optimize supply chains. Natural scientists such as geologists, biologists or experimental physicists also use Data Science to make their observations with the aim of gaining knowledge. Engineers want to better understand the situation and relationships between machinery or vehicles, and medical professionals are interested in better diagnostics and medication for their patients.

In order to support a specific department with his / her knowledge of data, tools and analysis methods, every data scientist needs a minimum of the appropriate skills. Anyone who wants to make analyzes for buyers, engineers, natural scientists, physicians, lawyers or other interested parties must also be able to understand the people’s profession.

Engere Data Science Definition

While the Data Science pioneers have long established and highly specialized teams, smaller companies are still looking for the Data Science Allrounder, which can take over the full range of tasks from the access to the database to the implementation of the analytical application. However, companies with specialized data experts have long since distinguished Data Scientists, Data Engineers and Business Analysts. Therefore, the definition of Data Science and the delineation of the abilities that a data scientist should have, varies between a broader and a more narrow demarcation.


A closer look at the more narrow definition shows, that a Data Engineer takes over the data allocation, the Data Scientist loads it into his tools and runs the data analysis together with the colleagues from the department. According to this, a Data Scientist would need no knowledge of databases or APIs, neither an expertise would be necessary …

In my experience, DataScience is not that narrow, the task spectrum covers more than just the core area. This misunderstanding comes from Data Science courses and – for me – I should point to the overall picture of Data Science again and again. In courses and seminars, which want to teach Data Science as a discipline, the focus will of course be on the core area: programming, tools and methods from mathematics & statistics.

Benjamin Aunkofer

Data Science Knowledge Stack – Was ein Data Scientist können muss

/in , , , , , , , /by

Was muss ein Data Scientist können? Diese Frage wurde bereits häufig gestellt und auch häufig beantwortet. In der Tat ist man sich mittlerweile recht einig darüber, welche Aufgaben ein Data Scientist für Aufgaben übernehmen kann und welche Fähigkeiten dafür notwendig sind. Ich möchte versuchen, diesen Konsens in eine Grafik zu bringen: Ein Schichten-Modell, ähnlich des OSI-Layer-Modells (welches übrigens auch jeder Data Scientist kennen sollte).
Ich gebe Einführungs-Seminare in Data Science für Kaufleute und Ingenieure und bei der Erläuterung, was wir in den Seminaren gemeinsam theoretisch und mit praxisnahen Übungen erarbeiten müssen, bin ich auf die Idee für dieses Schichten-Modell gekommen. Denn bei meinen Seminaren fängt es mit der Problemstellung bereits an, ich gebe nämlich Seminare für Data Science für Business Analytics mit Python. Also nicht beispielsweise für medizinische Analysen und auch nicht mit R oder Julia. Ich vermittle also nicht irgendein Data Science, sondern eine ganz bestimmte Richtung.

Ein Data Scientist muss bei jedem Data Science Vorhaben Probleme auf unterschiedlichsten Ebenen bewältigen, beispielsweise klappt der Datenzugriff nicht wie geplant oder die Daten haben eine andere Struktur als erwartet. Ein Data Scientist kann Stunden damit verbringen, seinen eigenen Quellcode zu debuggen oder sich in neue Data Science Pakete für seine ausgewählte Programmiersprache einzuarbeiten. Auch müssen die richtigen Algorithmen zur Datenauswertung ausgewählt, richtig parametrisiert und getestet werden, manchmal stellt sich dabei heraus, dass die ausgewählten Methoden nicht die optimalen waren. Letztendlich soll ein Mehrwert für den Fachbereich generiert werden und auch auf dieser Ebene wird ein Data Scientist vor besondere Herausforderungen gestellt.

english-flagRead this article in English:
“Data Science Knowledge Stack – Abstraction of the Data Scientist Skillset”

Data Science Knowledge Stack

Mit dem Data Science Knowledge Stack möchte ich einen strukturierten Einblick in die Aufgaben und Herausforderungen eines Data Scientists geben. Die Schichten des Stapels stellen zudem einen bidirektionalen Fluss dar, der von oben nach unten und von unten nach oben verläuft, denn Data Science als Disziplin ist ebenfalls bidirektional: Wir versuchen gestellte Fragen mit Daten zu beantworten oder wir schauen, welche Potenziale in den Daten liegen, um bisher nicht gestellte Fragen zu beantworten.

Der Data Science Knowledge Stack besteht aus sechs Schichten:

Database Technology Knowledge

Ein Data Scientist arbeitet im Schwerpunkt mit Daten und die liegen selten direkt in einer CSV-Datei strukturiert vor, sondern in der Regel in einer oder in mehreren Datenbanken, die ihren eigenen Regeln unterliegen. Insbesondere Geschäftsdaten, beispielsweise aus dem ERP- oder CRM-System, liegen in relationalen Datenbanken vor, oftmals von Microsoft, Oracle, SAP oder eine Open-Source-Alternative. Ein guter Data Scientist beherrscht nicht nur die Structured Query Language (SQL), sondern ist sich auch der Bedeutung relationaler Beziehungen bewusst, kennt also auch das Prinzip der Normalisierung.

Andere Arten von Datenbanken, sogenannte NoSQL-Datenbanken (Not only SQL)  beruhen auf Dateiformaten, einer Spalten- oder einer Graphenorientiertheit, wie beispielsweise MongoDB, Cassandra oder GraphDB. Einige dieser Datenbanken verwenden zum Datenzugriff eigene Programmiersprachen (z. B. JavaScript bei MongoDB oder die graphenorientierte Datenbank Neo4J hat eine eigene Sprache namens Cypher). Manche dieser Datenbanken bieten einen alternativen Zugriff über SQL (z. B. Hive für Hadoop).

Ein Data Scientist muss mit unterschiedlichen Datenbanksystemen zurechtkommen und mindestens SQL – den Quasi-Standard für Datenverarbeitung – sehr gut beherrschen.

Data Access & Transformation Knowledge

Liegen Daten in einer Datenbank vor, können Data Scientists einfache (und auch nicht so einfache) Analysen bereits direkt auf der Datenbank ausführen. Doch wie bekommen wir die Daten in unsere speziellen Analyse-Tools? Hierfür muss ein Data Scientist wissen, wie Daten aus der Datenbank exportiert werden können. Für einmalige Aktionen kann ein Export als CSV-Datei reichen, doch welche Trennzeichen und Textqualifier können verwendet werden? Eventuell ist der Export zu groß, so dass die Datei gesplittet werden muss.
Soll eine direkte und synchrone Datenanbindung zwischen dem Analyse-Tool und der Datenbank bestehen, kommen Schnittstellen wie REST, ODBC oder JDBC ins Spiel. Manchmal muss auch eine Socket-Verbindung hergestellt werden und das Prinzip einer Client-Server-Architektur sollte bekannt sein. Auch mit synchronen und asynchronen Verschlüsselungsverfahren sollte ein Data Scientist vertraut sein, denn nicht selten wird mit vertraulichen Daten gearbeitet und ein Mindeststandard an Sicherheit ist zumindest bei geschäftlichen Anwendungen stets einzuhalten.

Viele Daten liegen nicht strukturiert in einer Datenbank vor, sondern sind sogenannte unstrukturierte oder semi-strukturierte Daten aus Dokumenten oder aus Internetquellen. Auch hier haben wir es mit Schnittstellen zutun, ein häufiger Einstieg für Data Scientists stellt beispielsweise die Twitter-API dar. Manchmal wollen wir Daten in nahezu Echtzeit streamen, beispielsweise Maschinendaten. Dies kann recht anspruchsvoll sein, so das Data Streaming beinahe eine eigene Disziplin darstellt, mit der ein Data Scientist schnell in Berührung kommen kann.

Programming Language Knowledge

Programmiersprachen sind für Data Scientists Werkzeuge, um Daten zu verarbeiten und die Verarbeitung zu automatisieren. Data Scientists sind in der Regel keine richtigen Software-Entwickler, sie müssen sich nicht um Software-Sicherheit oder -Ergonomie kümmern. Ein gewisses Basiswissen über Software-Architekturen hilft jedoch oftmals, denn immerhin sollen manche Data Science Programme in eine IT-Landschaft integriert werden. Unverzichtbar ist hingegen das Verständnis für objektorientierte Programmierung und die gute Kenntnis der Syntax der ausgewählten Programmiersprachen, zumal nicht jede Programmiersprache für alle Vorhaben die sinnvollste ist.

Auf dem Level der Programmiersprache gibt es beim Arbeitsalltag eines Data Scientists bereits viele Fallstricke, die in der Programmiersprache selbst begründet sind, denn jede hat ihre eigenen Tücken und Details entscheiden darüber, ob eine Analyse richtig oder falsch abläuft: Beispielsweise ob Datenobjekte als Kopie oder als Referenz übergeben oder wie NULL-Werte behandelt werden.

Data Science Tool & Library Knowledge

Hat ein Data Scientist seine Daten erstmal in sein favorisiertes Tool geladen, beispielsweise in eines von IBM, SAS oder in eine Open-Source-Alternative wie Octave, fängt seine Kernarbeit gerade erst an. Diese Tools sind allerdings eher nicht selbsterklärend und auch deshalb gibt es ein vielfältiges Zertifizierungsangebot für diverse Data Science Tools. Viele (wenn nicht die meisten) Data Scientists arbeiten überwiegend direkt mit einer Programmiersprache, doch reicht diese alleine nicht aus, um effektiv statistische Datenanalysen oder Machine Learning zu betreiben: Wir verwenden Data Science Bibliotheken, also Pakete (Packages), die uns Datenstrukturen und Methoden als Vorgabe bereitstellen und die Programmiersprache somit erweitern, damit allerdings oftmals auch neue Tücken erzeugen. Eine solche Bibliothek, beispielsweise Scikit-Learn für Python, ist eine in der Programmiersprache umgesetzte Methodensammlung und somit ein Data Science Tool. Die Verwendung derartiger Bibliotheken will jedoch gelernt sein und erfordert für die zuverlässige Anwendung daher Einarbeitung und Praxiserfahrung.

Geht es um Big Data Analytics, also die Analyse von besonders großen Daten, betreten wir das Feld von Distributed Computing (Verteiltes Rechnen). Tools (bzw. Frameworks) wie Apache Hadoop, Apache Spark oder Apache Flink ermöglichen es, Daten zeitlich parallel auf mehren Servern zu verarbeiten und auszuwerten. Auch stellen diese Tools wiederum eigene Bibliotheken bereit, für Machine Learning z. B. Mahout, MLlib und FlinkML.

Data Science Method Knowledge

Ein Data Scientist ist nicht einfach nur ein Bediener von Tools, sondern er nutzt die Tools, um seine Analyse-Methoden auf Daten anzuwenden, die er für die festgelegten Ziele ausgewählt hat. Diese Analyse-Methoden sind beispielweise Auswertungen der beschreibenden Statistik, Schätzverfahren oder Hypothesen-Tests. Etwas mathematischer sind Verfahren des maschinellen Lernens zum Data Mining, beispielsweise Clusterung oder Dimensionsreduktion oder mehr in Richtung automatisierter Entscheidungsfindung durch Klassifikation oder Regression.

Maschinelle Lernverfahren funktionieren in der Regel nicht auf Anhieb, sie müssen unter Einsatz von Optimierungsverfahren, wie der Gradientenmethode, verbessert werden. Ein Data Scientist muss Unter- und Überanpassung erkennen können und er muss beweisen, dass die Vorhersageergebnisse für den geplanten Einsatz akkurat genug sind.

Spezielle Anwendungen bedingen spezielles Wissen, was beispielsweise für die Themengebiete der Bilderkennung (Visual Computing) oder der Verarbeitung von menschlicher Sprache (Natural Language Processiong) zutrifft. Spätestens an dieser Stelle öffnen wir die Tür zum Deep Learning.

Fachexpertise

Data Science ist kein Selbstzweck, sondern eine Disziplin, die Fragen aus anderen Fachgebieten mit Daten beantworten möchte. Aus diesem Grund ist Data Science so vielfältig. Betriebswirtschaftler brauchen Data Scientists, um Finanztransaktionen zu analysieren, beispielsweise um Betrugsszenarien zu erkennen oder um die Kundenbedürfnisse besser zu verstehen oder aber, um Lieferketten zu optimieren. Naturwissenschaftler wie Geologen, Biologen oder Experimental-Physiker nutzen ebenfalls Data Science, um ihre Beobachtungen mit dem Ziel der Erkenntnisgewinnung zu machen. Ingenieure möchten die Situation und Zusammenhänge von Maschinenanlagen oder Fahrzeugen besser verstehen und Mediziner interessieren sich für die bessere Diagnostik und Medikation bei ihren Patienten.

Damit ein Data Scientist einen bestimmten Fachbereich mit seinem Wissen über Daten, Tools und Analyse-Methoden ergebnisorientiert unterstützen kann, benötigt er selbst ein Mindestmaß an der entsprechenden Fachexpertise. Wer Analysen für Kaufleute, Ingenieure, Naturwissenschaftler, Mediziner, Juristen oder andere Interessenten machen möchte, muss eben jene Leute auch fachlich verstehen können.

Engere Data Science Definition

Während die Data Science Pioniere längst hochgradig spezialisierte Teams aufgebaut haben, suchen beispielsweise kleinere Unternehmen eher den Data Science Allrounder, der vom Zugriff auf die Datenbank bis hin zur Implementierung der analytischen Anwendung das volle Aufgabenspektrum unter Abstrichen beim Spezialwissen übernehmen kann. Unternehmen mit spezialisierten Daten-Experten unterscheiden jedoch längst in Data Scientists, Data Engineers und Business Analysts. Die Definition für Data Science und die Abgrenzung der Fähigkeiten, die ein Data Scientist haben sollte, schwankt daher zwischen der breiteren und einer engeren Abgrenzung.

Die engere Betrachtung sieht vor, dass ein Data Engineer die Datenbereitstellung übernimmt, der Data Scientist diese in seine Tools lädt und gemeinsam mit den Kollegen aus dem Fachbereich die Datenanalyse betreibt. Demnach bräuchte ein Data Scientist kein Wissen über Datenbanken oder APIs und auch die Fachexpertise wäre nicht notwendig…

In der beruflichen Praxis sieht Data Science meiner Erfahrung nach so nicht aus, das Aufgabenspektrum umfasst mehr als nur den Kernbereich. Dieser Irrtum entsteht in Data Science Kursen und auch in Seminaren – würde ich nicht oft genug auf das Gesamtbild hinweisen. In Kursen und Seminaren, die Data Science als Disziplin vermitteln wollen, wird sich selbstverständlich auf den Kernbereich fokussiert: Programmierung, Tools und Methoden aus der Mathematik & Statistik.

Marta Fajlhauer

Is Data Science the new Statistics?

/in , , , , , , , /by

Table of Contents

1 Introduction

2 Emerging of Data Science

3 Big data technologies

4 Two data worlds: Predictive vs inferential statistics

5 How to study data science

6 Conclusions

7 References

Introduction

As a student of Statistics and the winner of Data Science Scholarship I am often surrounded by computer scientists, mathematicians, physicists and of course statisticians. During conversation, I was asked questions such as “So what actually do I do? What is Data Science?”. These are some very difficult questions and as like you will see during reading this document many before me tried to answer those questions. There is a dispute between statisticians and computer scientists what is the origin of data science and who should teach it. According to the Institute of Mathematical Statistics in the: “The IMS presidential address: let us own data science” we can find a simple recipe for data scientist. [1]

“Putting the traits of Turner and Carver together gives a good portrait of a data scientist:

  • Statistics (S)
  • Domain/Science knowledge (D)
  • Computing (C)
  • Collaboration/teamwork (C)
  • Communication to outsiders (C)

That is, data science = SDCCC = S DC3

However, despite all the challenges that I will need to overcome in answering those questions I will try to do it. I will refer to ideas from several reputable sources, in which I will also tell you: what is in the data science that I am really fascinated about? What is magical in this creation of statistics and computer science that I am drawn to?

Emerging of Data Science

On Tuesday, the 8th of September 2015, University of Michigan announced the 100 million dollars “Data Science Initiative” (DSI), hired 35 new faculty members. On the DSI website we can read about this initiative:

“This coupling of scientific discovery and practice involves the collection, management, processing, analysis, visualisation, and interpretation of vast amounts of heterogeneous data associated with a diverse array of scientific, translational and interdisciplinary applications”2

But that sounds like a bread and butter for statisticians. So, is it really a new creation or is it something that exists for many years but it didn’t sound so sexy as data science? In the article written by Karl Broman, (the University of Wisconsin) we can read:

“When physicists do mathematics, they’re don’t say they’re doing “number science”. They’re doing math. If you’re analyzing data, you’re doing statistics. You can call it data science or informatics or analytics or whatever, but it ‘s still statistics. If you say that one kind of data analysis is statistics and another kind is not, you’re not allowing innovation. We need to define the field broadly. You may not like what some statisticians do. You may feel they don’t share your values. They may embarrass you. But that shouldn’t lead us to abandon the term “statistics”.

Reading the definition of data science on the Data Science Association’s “Professional Code of Conduct”:

“Data scientist means a professional who uses scientific methods to liberate and create meaning from raw data”

These sound like K. Browman maybe right. Maybe I should go on MSc Statistics like many before me did. Maybe Data Science is simply a new sexy name for statistician only data is big, technology more advanced rather than it used to be so you need to have programming skills to handle the data. Maybe let say loudly data science is a modern version of statistics? But maybe not? Because we can also find statements like the following:

“Statistics is the least important part of data science”. [3]

Further, we can read:

“There ‘s so, much that goes on with data that is about computing, not statistics. I do think it would be fair to consider statistics (which includes sampling, experimental design, and data collection as well as data analysis (which itself includes model building, visualization, and model checking as well as inference)) as a subset of data science. . . .”.[3]

So maybe people from computer science are right. Maybe I should go and study programming and forget about expanding my knowledge in statistics? After all, we all know that computer science always had much bigger funding and having MSc computer science was always like a magic star for employers. What should I do? Let me research further.

Big data technologies

Is the data size important to distinguish between data science and statistics? Going back to the “Let us own data science” article we can read that a statistician, Hollerith, invented the punched card reader to allow e cient compilation of a US census, the first elements of machine learning. So, no, machine learning is not an invention of computer scientists. It was well known for statistician for decades already. What about different techniques used in DOE (Design of Experiments) or sampling methods to decrease the sample size. If the data used by statisticians would be only small they wouldn’t have to discover methods such PCA (Principle component analysis) or dimensionality reduction techniques. So, no, data can be big and/or small for statisticians, so what is the difference between data science and statistics and what department should I choose?

When I spoke to computer scientists they try to convince me to choose computer science department. Their reasons being that there are many different programmes that I need to know to deal with large datasets. For instance: Java, Hadoop, SQL, Python, and much more. Moreover, programming can only be taught to the best standard through computer science courses Is it true? Can’t we do the same calculations using statistical software such as R, SAS or even Matlab? But on the other hand, doesn’t the newest technology always work faster? And if so, wouldn’t be better to use the newest technology when we program and write loops?

But, I don’t want to underestimate the effort made by statisticians and data analyst over last 50 years in developing statistical programmes. Their efforts have resulted in the emergence of today’s technology. Early statistical packages such as SPSS or Minitab (from 1960’s) allowed to develop more advanced programmes having roots in mini computer era such as STATA or my favourite R which in turn allowed progress to advanced technology even further and create Python, Hadoop, SQL and so on. Becker and Chambers (with S) and later Ihaka, Gentleman, and members of the R Core team (with R) worked on developing the statistical software. These names should be convincing about how powerful statistical programming languages can be. Many operations that we can do in Hadoop or SQL we can also do easily in R.

Two data worlds: Predictive vs inferential statistics

So maybe Data Science is a creature merged by statisticians working on computer science department? Maybe there are two different approaches to statistics: mathematical statistics and computer science statistics and the computer science statisticians are data scientists because according to Yanir Seroussi in his blog:

“A successful data scientist needs to be able to “become one with the data” by exploring it and applying rigorous statistical analysis (right-hand side of the continuum). But good data scientists also understand what it takes to deploy production systems, and are ready to get their hands dirty by writing code that cleans up the data or performs core system functionality (lefthand side of the continuum). Gaining all these skills takes time.”[4]

Okay, so my reasoning that some statisticians work on computer science department is right, as well as there exists subject like computational statistics, so maybe I should go for computer science department but study statistics.

In fact, I am not the first one to arrive at the conclusion. Everything started from a confession made by John Tukey in “The Future of Data Analysis” article published in “The Annals of Mathematical Statistics” :

For a long time, I have thought I was a statistician, interested in inferences from the particular to the general. But as I have watched mathematical statistics evolve, I have had cause to wonder and to doubt. … All in all I have come to feel that my central interest is in data analysis, which I take to include, among other things: procedures for analyzing data, techniques for interpreting the results of such procedures, ways of planning the gathering of data to make its analysis easier, more precise or more accurate, and all the machinery and results of (mathematical) statistics which apply to analyzing data

If I am right then above confession was a critical moment. The time when mathematical statistics become more inferential and computational statistics concentrated more on predictive statistics. Applied statisticians working on predictive analytics that are more interested in applying the knowledge rather than developing long proofs decided to move on computer science department.

Additionally, the following is crucial discussion made by Leo Biermann in his paper published in Statistical Science titled “Statistical modelling: the two cultures”. It enables us to understand and differentiate views from both types of statistician, namely mathematical and statistical.

Statistics starts with data. Think of the data as being generated by a black box in which a vector of input variables x (independent variables) go in one side, and on the other side the response variables y come out. Inside the black box, nature functions to associate the predictor variables with the response variables … There are two goals in analyzing the data:

  • Prediction. To be able to predict what the responses are going to be to future input variables
  • InferenceTo [infer] how nature is associating the response variables to the input variables.”

Furthermore, in the same dispute we can read:

“The statistical community has been committed to the almost exclusive use of [generative] models. This commitment has led to irrelevant theory, questionable conclusions, and has kept statisticians from working on a large range of interesting current problems. [Predictive] modeling, both in theory and practice, has developed rapidly in fields outside statistics. It can be used both on large complex data sets and as a more accurate and informative alternative to data modeling on smaller data sets. If our goal as a field is to use data to solve problems, then we need to move away from exclusive dependence on [generative] models …”

So, we can say that Data Science evolved from Predictive Analytics which in turn evolved from Statistics but it becomes separate science. Tukey and Wilk 1969 compared this new science to established sciences and further circumscribed the role of Statistics within it:

“ … data analysis is a very di cult field. It must adapt itself to what people can and need to do with data. In the sense that biology is more complex than physics, and the behavioural sciences are more complex than either, it is likely that the general problems of data analysis are more complex than those of all three. It is too much to ask for close and effective guidance for data analysis from any highly formalized structure, either now or in the near future. Data analysis can gain much from formal statistics, but only if the connection is kept adequately loose”

How to study data science

So, what is exactly predictive analytics culture? I think that everyone who used Kaggle competition before can agree with me that description of common task framework (CTF) formulated by Marc Liberman in 2009 is a perfect description of Kaggle competitions, and hackathons events; where latter has worked as training sessions for newbies in the data world. An instance of the CTF has these ingredients:

  1. A publicly available training data set involving, for each observation, a list of (possibly many) feature measurements, and a class label for that observation.
  2. A set of enrolled competitors whose common task is to infer a class prediction rule from the training data.
  3. A scoring referee, to which competitors can submit their prediction rule. The referee runs the prediction rule against a testing dataset which is sequestered behind a Chinese wall. The referee objectively and automatically reports the score (prediction accuracy) achieved by the submitted rule

Kaggle competitions are not only training platforms for newbies like me but also very challenging statistical competitions where experienced statisticians can win “pocket money”. A famous example is the Netflix Challenge where the common task was to predict Netflix user movie selection. The winning team (which included ATT Statistician Bob Bell) won 1 mln dollars.

Comparing modules that are available on master in data science at University of Berkley[6]:

  1. Both
  • Applied machine learning
  • Experiments and causality
  1. Statistics
  • Research design and application for data and analysis
  • Statistics for Data Science
  • Behind the data: humans and values
  • Statistical methods for discrete response, Time Series and panel data
  • Data visualisation
  1. Computer Science
  • Python for Data Science
  • Storing and Retrieving Data
  • Scalling up! Really Big Data
  • Machine Learning at scale
  • Natural Language Processing with Deep Learning

We can really see that data science is a subject that demands skills from both computer science and statistics. So, it is another confirmation for me that it is the best time to change department for my postgraduate study, that is, to study statistics on computer science department.

In the 50 Years of Data Science article we can read: “The activities of Greater Data Science are classified into 6 divisions:

  1. Data exploration and preparation
  2. Data representation and transformation
  3. Computing with data
  4. Data visualization and presentation
  5. Data Modelling
  6. Science about data science [5]

I will quickly go through all of them using my Ebola research example, this required using machine learning on time series data.

  1. The most demanding part. Many people told me before starting this project that: collecting, cleaning, wrangling and preparing data take 60% of all the time that you need to spend on data science project. I didn’t realise how much this 60% means in real time. I didn ‘t realise that the 60 percent will take so much time and that after this I will be exhausted. Exhausted but ready for the next step.
  2. This point is actually part of the first one, or maybe just like many other things in statistics: everything is one huge connected bunch.Data that you can find can be very nice, well behaving, written in CSV or JSON or any other format file that you can quickly download and use, but what if not? What if your data is ‘dirty’and not stored as a file (e.g. only appear on a website)? What if data is coded? Do you need to decode it?
  3. The even bigger challenge, but what a fun? You need to know a few different programming languages or least as I do know a little bit of R, a little bit of Python, quite well Tableau and Excel. So you can use different program in different scenarios or for different tasks. For example, using Panda to do EDA and ggplot 2 to do data vis.
  4. Graphs are pretty, right? If you are still reading my article, I bet you know what is heat map, spatial vis in big cities or different infographics. Surely, I would like to highlight, that we respect only the ones that are not only pretty but also valid. Nevertheless, time that is required to create these visualisations is another matter.
  5. The data modelling, finally? I don’t need to say a lot about this. All forms of inferential and predictive analytic are allowed and accepted.
  6. My favourite part, not the end yet. All the conferences and meetups that I can attend on. All the seminars where we all present our current projects.

Conclusions

After graduation, I will be graduated Statistician. Even more, I will be a mathematical statistician whom mostly during degree dealt with inferential statistics. On the other hand, winning data science scholarship gave me exposure to predictive analytic which I highly enjoyed. Therefore, for my next stage, I will just change my department and concentrate more on predictive analytic. There are many statisticians working on computer science department. They possess both statistical knowledge and advanced software engineering skills, they are called data scientists. It would be a pleasure for me to join them. I don’t mind if it will be MSc. Computer Science, MSc. Data Science, MSc. Big Data or whatever the name will be. I do mind to have sufficient exposure to deal with “dirty” data using statistical modelling and machine learning using modern technology. This is what data science is for me. Maybe for you, it will be something else. Maybe you will be more satisfied with expanding massively programming skills. But for me, programming is a tool, modern technology is my friend and my bread and butter will be predictive analytic.

References

  1. IMS Presidential Address: Let us own data science
  2. Data science is statistics
  3. A Gelman, Columbia University
  4. Yanir Seroussi: What is data Science?
  5. 50 Years Data Science
  6. Curriculum: data science@Berkley
Benjamin Aunkofer

Unternehmen brauchen eine Datenstrategie

/in , , , , , , , , , , /by

Viele Unternehmen stecken gerade in der Digitalisierung fest, digitalisieren Prozesse und Dokumente, vernetzen immer mehr Maschinen und Endgeräte, und generieren dabei folglich immer mehr Daten. Aber auch ungeachtet der aktuellen Digitalisierungs- und Vernetzungsinitiativen verfügen Unternehmen bereits längst über einen wahren Datenschatz in Ihren ERP-, CRM- und sonstigen IT-Systemen. Hinzu kommt ein beinahe unerschöpfliches Datenpotenzial aus externen Quellen hinzu, insbesondere dem Social Media, den Finanzportalen und behördlichen Instituten (Open Data).

Nur die wenigsten Unternehmen – jene dürfen wir ohne Zweifel zu den Gewinnern der Digitalisierung zählen – verfügen über eine konkrete Strategie, wie Daten aus unternehmensinternen und -externen Datenquellen zur Geschäftsoptimierung genutzt werden können: Die Datenstrategie.

Was ist eine Datenstrategie?

Die Datenstrategie ist ein ausformulierter und zielorientierter Verfahrensplan, um Daten in Mehrwert zu verwandeln. Er bringt während seiner Formulierung alle nötigen Funktionsbereichen zusammen, also IT-Administratoren, kaufmännische Entscheider und natürlich Data Scientists bzw. Datenexperten (welche genaue Berufsbezeichnung auch immer damit verbunden sein mag).

Die Datenstrategie ist ein spezieller Business Plan zur gewinnorientierten Datennutzung. In ihr werden klare Ziele und Zeitvorgaben (kurz-, mittel-, langfristig) definiert, der voraussichtliche Ressourcen-Einsatz und die Rahmenbedingungen benannt. Dazu gehören sowohl die technischen (Hardware, Software) als auch die rechtlichen Rahmen (Datenschutz, Datensicherheit, Urheberrecht usw.). Die Datenstrategie die Herausforderungen nachvollziehbar heraus und stellt im Abgleich fest, ob die bestehende Belegschaft im aktuellen Zustand die nötigen Kapazitäten und Qualifikationen hat bzw. ob Maßnahmen zum Erwerb von Know-How (Qualifizierung, Recruiting) ergriffen werden sollten.

Wozu braucht ein Unternehmen eine Datenstrategie?

Viele Unternehmen – ich bin zumindest mit vielen solcher Unternehmen im Gespräch – wissen oft nicht, wie sie am Trend zur Datennutzung partizipieren können, bevor es der Wettbewerb tut bzw. man für neue Märkte unzureichend / zu spät vorbereitet ist. Sie wissen, dass es Potenziale für die Nutzung von Daten gibt, jedoch nicht, welche Tragweite derartige Projekte hinsichtlich des Einsatzes und des Ergebnisses haben werden. Diesen Unternehmen fehlt eine Datenstrategie als ein klarer Fahrplan, um über Datenanalyse die bestehenden Geschäfte zu optimieren. Und möglicherweise auch, um neue Geschäftsmöglichkeiten zu erschließen.

Demgegenüber steht eine andere Art von Unternehmen: Diese sind bereits seit Jahren in die Nutzung von Big Data eingestiegen und haben nun viele offene Baustellen, verschiedene neue Tools und eine große Vielfalt an Projektergebnissen. Einige dieser Unternehmen sehen sich nunmehr mit einer Komplexität konfrontiert, für die der Wunsch nach Bereinigung aufkommt. Hier dient die Datenstrategie zur Fokussierung der Ressourcen auf die individuell besten, d.h. gewinnträchtigsten bzw. nötigsten Einsatzmöglichkeiten, anstatt alle Projekte auf einmal machen.

Zusammenfassend kann demnach gesagt werden, dass eine Datenstrategie dazu dient, sich nicht in Big Data bzw. Data Science Projekte zu verrennen oder mit den falschen Projekten anzufangen. Die Strategie soll Frustration vermeiden und schon vom Ansatz her dafür sorgen, dass die nächst höhere Etage – bis hin zum Vorstand – Big Data Projekte nicht für sinnlos erklärt und die Budgets streicht.

Wie entsteht eine Datenstrategie?

Ein ganz wesentlicher Punkt ist, dass die Datenstrategie kein Dokument wird, welches mühsam nur für die Schublade erstellt wurde. Der Erfolg entsteht schließlich nicht auf schönen Strategiefolien, sondern aus zielgerichteter Hands-on-Arbeit. Zudem ist es erfolgskritisch, dass die Datenstrategie für jeden beteiligten Mitarbeiter verständlich ist und keine Beraterfloskeln enthält, jedoch fachlich und umsetzungsorientiert bleibt. Im Kern steht sicherlich in der Regel eine Analysemethodik (Data Science), allerdings soll die Datenstrategie alle relevanten Fachbereiche im Unternehmen mitnehmen und somit ein Gemeinschaftsgefühl (Wir-Gefühl) erschaffen, und keinesfalls die Erwartung vermitteln, die IT mache da schon irgendwas. Folglich muss die Datenstrategie gemeinschaftlich entwickelt werden, beispielsweise durch die Gründung eines Komitees, welches aus Mitarbeitern unterschiedlichster Hintergründe besetzt ist, die der Interdisziplinität gerecht wird. Eine entsprechend nötige Interdisziplinität des Teams bringt übrigens – das wird häufig verschwiegen – auch Nachteile mit sich, denn treffen die führenden Köpfe aus den unterschiedlichen Fachbereichen aufeinander, werden Vorschläge schnell abgehoben und idealistisch, weil sie die Erwartungen aller Parteien erfüllen sollen. Eine gute Datenstrategie bleibt jedoch auf dem Boden und hat realistische Ziele, sie orientiert sich an den Gegebenheiten und nicht an zukünftigen Wunschvorstellungen einzelner Visionäre.

Idealerweise wird die Entwicklung der Datenstrategie von jemanden begleitet, der sowohl Erfahrung in Verarbeitung von Daten als auch vom Business hat, und der über explizite Erfahrung mit Big Data Projekten verfügt. Gerade auch das Einbeziehen externer Experten ermöglicht, dass indirekt durch den Erfahrungseinfluss aus bereits gemachten Fehlern in anderen Unternehmen gelernt werden kann.

Mehr dazu im nächsten Artikel: Die fünf Schritte zur Datenstrategie!