Tag Archive for: Data Scientist

Benjamin Aunkofer

Data Security for Data Scientists & Co. – Infographic

/in , , , , , /by

Data becomes information and information becomes knowledge. For this reason, companies are nowadays also evaluated with regard to their data and their data quality. Furthermore, data is also the material that is needed for management decisions and artificial intelligence. For this reason, IT Security is very important and special consulting and auditing companies offer their own services specifically for the security of IT systems.

However, every Data Scientist, Data Analyst and Data Engineer rarely only works with open data, but rather intensively with customer data. Therefore, every expert for the storage and analysis of data should at least have a basic knowledge of Data Security and work according to certain principles in order to guarantee the security of the data and the legality of the data processing.

There are a number of rules and principles for data security that must be observed. Some of them – in our opinion the most important ones – we from DATANOMIQ have summarized in an infographic for Data Scientists, Data Analysts and Data Engineers. You can download the infographic here: DataSecurity_Infographic

Data Security for Data Scientists, Data Analysts and Data Engineers

Data Security for Data Scientists, Data Analysts and Data Engineers

Download Infographic as PDF

Infographic - Data Security for Data Scientists, Data Analysts and Data Engineers

Infographic – Data Security for Data Scientists, Data Analysts and Data Engineers

Matthew Simons

Select the Right career path between Software Developer and Data Scientist

/in , , /by

In today’s digital day and age, a software development career is one of the most lucrative ones. Custom software developers abound, offering all sorts of services for business organizations anywhere in the world. Software developers of all kinds, vendors, full-time staff, contract workers, or part-time workers, all are important members of the Information Technology community. 

There are different career paths to choose from in the world of software development. Among the most promising ones include a software developer career and a data scientist career. What exactly are these?

Software developers are the brainstorming, creative masterminds behind all kinds of computer programs. Although there may be some that focus on a specific app or program, others build giant networks or underlying systems, which power and trigger other programs. That’s why there are two classifications of a software developer, the app software developer, and the developers of systems software.

On the other hand, data scientists are a new breed of experts in analytical data with the technical skills to resolve complex issues, as well as the curiosity to explore what problems require solving. Data scientists, in any custom software development service, are part trend-spotter, part mathematicians, and part computer scientists. And, since they bestraddle both IT and business worlds, they’re highly in-demand and of course well-paid. 

When it comes to the field of custom software development and software development in general, which career is the most promising? Let’s find out. 

Data Science and Software Development, the Differences

Although both are extremely technical, and while both have the same sets of skills, there are huge differences in how these skills are applied. Thus, to determine which career path to choose from, let’s compare and find the most critical differences. 

The Methodologies

Data Science Methodology

There are different places in which a person could come into the data science pipeline. If they are gathering data, then they probably are called a data engineer, and they would be pulling data from different resources, cleaning and processing it, and storing it in a database. Usually, this is referred to as the ETL process or the extract, transform, and load. 

If they use data to create models and perform analysis, probably they’re called a ‘data analyst’ or a ‘machine learning engineer’. The critical aspects of this part of the pipeline are making certain that any models made don’t violate the underlying assumptions, and that they are driving worthwhile insights. 

Methodology in Software Development 

In contrast, the development of software makes use of the SDLC methodology or the software development life cycle. The workflow or cycle is used in developing and maintaining software. The steps are planning, implementing, testing, documenting, deploying, and maintaining. 

Following one of the different SDLC models, in theory, could lead to software that runs at peak efficiency and would boost any future development. 

The Approaches

Data science is a very process-oriented field The practitioners consume and analyze sets of data to understand a problem better and come up with a solution. Software development is more of approaching tasks with existing methodologies and frameworks. For example, the Waterfall model is a popular method that maintains every software development life cycle phase that should be completed and reviewed before going to the next. 

Some frameworks used in development include the V-shaped model, Agile, and Spiral. Simply, there is no equal data science process, although a lot of data scientists are within one of the approaches as part of the bigger team. Pure developers of the software have a lot of roles to fill outside data science, from front-end development to DevOps and infrastructure roles. 

Moreover, although data analytics pays well, the roles of software developers of all kinds are still higher in demand. Thus, if machine learning isn’t your thing, then you could spend your spare time in developing expertise in your area of interest instead. 

The Tools

The wheelhouse of a data scientist has data analytics tools, machine learning, data visualization, working with databases, and predictive modeling. If you use plenty of data ingestion and storage they probably would use MongoDB, Amazon S3, PostgreSQL, or something the same. For building a model, there’s a great chance that they would be working with Scikit-learn or Statsmodels. 

Big data distributed processing needs Apache Spark. Software engineers use software to design and analyze tools, programming languages, software testing, web apps tools, and so on. With data science, many depend on what you’re attempting to accomplish. For actually creating TextWrangler, code Atom, Emacs, Visual Code Studio, and Vim are popular. 

Django by Python, Ruby on Rails, and Flask see plenty of use in the backend web development world. Vue.js emerged recently as one of the best ways of creating lightweight web apps, and similarly for AJAX when creating asynchronous-updating, creating dynamic web content. Everyone must know how to utilize a version control system like GitHub for instance. 

The Skills

To become a data scientist, some of the most important things to know include machine learning, programming, data visualization, statistics, and the willingness to learn. Various positions may need more than these skills, but it’s a safe bet to say that these are the bare minimum when you pursue a data science career. 

Often, the necessary skills to be a developer of the software will be a little more intangible. The ability of course to program and code in various programming languages is required, but you should also be able to work well in development teams, resolve an issue, adapt to various scenarios, and should be willing to learn. This again isn’t an exhaustive list of skills, but these certainly would serve you well if you are interested in this career. 

Conclusion

You should, at the end of the day must choose a career path that’s based on your strengths and interests. The salaries of data scientists and software developers  are the same to an average at least. However, before choosing which is better for you, consider experimenting with various projects and interact with different aspects of the business to determine where your skills and personality best fits in since that is where you’ll grow the most in the future.

Michael Lyamm

Closing the AI-skills gap with Upskilling

/in , /by

Closing the AI-skills gap with Upskilling

Artificial Intelligent or as it is fancily referred as AI, has garnered huge popularity worldwide.  And given the career prospects it has, it definitely should. Almost everyone interested in technology sector has them rushing towards it, especially young and motivated fresh computer science graduates. Compared to other IT-related jobs AI pays way higher salary and have opportunities. According to a Glassdoor report, Data Scientist, one of the many related jobs, is the number one job with good salary, job openings and more. AI-related jobs include Data Scientists, Analysts, Machine Learning Engineer, NLP experts etc.

AI has found applications in almost every industry and thus it has picked up demand. Home assistants – Siri, Ok Google, Amazon Echo — chatbots, and more some of the popular applications of AI.

Increasing adoption of AI across Industry

The advantages of AI like increased productivity has increased its adoption among companies. According to Gartner, 37 percent of enterprise currently use AI in one way or the other. In fact, in the last four year adoption of AI technologies among companies has increased by 270 percent. In telecommunications, for instance, 52 percent of companies have chatbots deployed for better and smoother customer experience. Now, about 49 percent of businesses are now on their way to alter business models to integrate and adopt AI-driven processes. Further, industry leaders have gone beyond and voiced their concerns about companies that are lagging in AI adoption.

Unfortunately, it has been extremely difficult for employers to find right skilled or qualified candidates for AI-related positions. A reports suggests that there are total 300,000 AI professionals are available worldwide, while there’s demand for millions. In a recent survey conducted by Ernst & Young, 51 percent AI professionals told that lack of talent was the biggest impediment in AI adoption.

Further, O’Reilly, in 2018 conducted a survey, which found the lack of AI skills, among other things, was the major reason that was holding companies back from implementing AI.
The major reason for this is the lack of skills among people who aspire to get into AI-related jobs. According to a report, there demand for millions for jobs in AI. However, only a handful of qualified people are available.

Bridging the skill gap in AI-related jobs

Top companies and government around the world have taken up initiatives to close this gap. Google and Amazon, for instance, have dedicated facilities which trains in AI skills.  Google’s Brain Toronto is a dedicated facility to expand their talent in AI.  Similarly, Amazon has facility near University of Cambridge which is dedicated to AI. Most companies either already have a facility or are in the process of setting up one.

In addition to this, governments around the world are also taking initiatives to address the skill gap. For instance, government across the world are pushing towards AI advancement and are develop collaborative plans which aims at delivering more AI skilled professionals. Recently, the white house launched ai.gov which is further helping to promote AI in the US. The website will offer updates related to AI projects across different sectors.

Other than these, companies have taken this upon themselves to reskills their employees and prepare them for future roles. According to a report from Towards Data Science, about 63 percent of companies have in-house training programs to train employees in AI-related skills.

Overall, though there is demand for AI professionals, lack of skilled talent is a major problem.

Roles in Artificial Intelligence
Artificial Intelligence is the most dominant role for which companies hire across artificial Intelligence. Other than that, following are some of the popular roles:

  1. Machine learning Engineer: These are the people who make machines learn with complex algorithms. On advance level, Machine learning engineers are required to have good knowledge of computer vision. According to Indeed, in the last year, demand for Machine Learning Engineer has grown by 344 percent.
  2. NLP Experts: These experts are equipped with the understanding of making machines computer understand human language. Their expertise includes knowledge of how machines understand human language. Text-to-speech technologies are the common areas which require NLP experts. Demand for engineers who can program computers to understand human speech is growing continuously. It was the fast growing skills in Upwork’s list of in-demand freelancing skills. In Q4, 2016, it had grown 200 percent and since then has been on continuously growing.
  3. Big Data Engineers: This is majorly an analytics role. These gather huge amount of data available from sources and analyze it to derive insights and understand patter, which may be further used for machine learning, prediction modelling, natural language processing. In Mckinsey annual report 2018, it had reported that there was shortage of 190,000 big data professionals in the US alone.

Other roles like Data Scientists, Analysts, and more also in great demand. Then, again due to insufficient talent in the market, companies are struggling to hire for these roles.

Self-learning and upskilling
Artificial Intelligence is a continuously growing field and it has been advancing at a very fast pace, and it makes extremely difficult to keep up with in-demand skills. Hence, it is imperative to keep yourself up with demand of the industry, or it is just a matter of time before one becomes redundant.

On an individual level, learning new skills is necessary. One has to be agile and keep learning, and be ready to adapt new technologies. For this, AI training programs and certifications are ideal.  There are numerous AI programs which individuals can take to further learn new skills. AI certifications can immensely boost career opportunities. Certification programs offer a structured approach to learning which benefits in learning mostly practical and executional skills while keeping fluff away. It is more hands-on. Plus, certifications programs qualify only when one has passed practical test which is very advantageous in tech. AI certifications like AIE (Artificial Intelligence Engineer) are quite popular.

Online learning platforms also offer good a resource to learn artificial intelligence. Most schools haven’t yet adapted their curriculum to skill for AI, while most universities and grad schools are in their way to do so. In the meantime, online learning platforms offer a good way to learn AI skills, where one can start from basic and reach to advance skills.

Benjamin Aunkofer

Business Intelligence Organizations

/in , , , /by

I am often asked how the Business Intelligence department should be set up and how it should interact and collaborate with other departments. First and foremost: There is no magic recipe here, but every company must find the right organization for itself.

Before we can talk about organization of BI, we need to have a clear definition of roles for team members within a BI department.

A Data Engineer (also Database Developer) uses databases to save structured, semi-structured and unstructured data. He or she is responsible for data cleaning, data availability, data models and also for the database performance. Furthermore, a good Data Engineer has at least basic knowledge about data security and data privacy. A Data Engineer uses SQL and NoSQL-Technologies.

A Data Analyst (also BI Analyst or BI Consultant) uses the data delivered by the Data Engineer to create or adjust data models and implementing business logic in those data models and BI dashboards. He or she needs to understand the needs of the business. This job requires good communication and consulting skills as well as good developing skills in SQL and BI Tools such like MS Power BI, Tableau or Qlik.

A Business Analyst (also Business Data Analyst) is a person form any business department who has basic knowledge in data analysis. He or she has good knowledge in MS Excel and at least basic knowledge in data analysis and BI Tools. A Business Analyst will not create data models in databases but uses existing data models to create dashboards or to adjust existing data analysis applications. Good Business Analyst have SQL Skills.

A Data Scientist is a Data Analyst with extended skills in statistics and machine learning. He or she can use very specific tools and analytical methods for finding pattern in unknow or big data (Data Mining) or to predict events based on pattern calculated by using historized data (Predictive Analytics). Data Scientists work mostly with Python or R programming.

Organization Type 1 – Central Approach (Data Lab)

The first type of organization is the data lab approach. This organization form is easy to manage because it’s focused and therefore clear in terms of budgeting. The data delivery is done centrally by experts and their method and technology knowledge. Consequently, the quality expectation of data delivery and data analysis as well as the whole development process is highest here. Also the data governance is simple and the responsibilities clearly adjustable. Not to be underestimated is the aspect of recruiting, because new employees and qualified applicants like to join a central team of experts.

However, this form of organization requires that the company has the right working attitude, especially in the business intelligence department. A centralized business intelligence department acts as a shared service. Accordingly, customer-oriented thinking becomes a prerequisite for the company’s success – and customers here are the other departments that need access to the capacities of those centralized data experts. Communication boundaries must be overcome and ways of simple and effective communication must be found.

Organization Type 2 – Stakeholder Focus Approach

Other companies want to shift more responsibility for data governance, and especially data use and analytics, to those departments where data plays a key role right now. A central business intelligence department manages its own projects, which have a meaning for the entire company. The specialist departments, which have a special need for data analysis, have their own data experts who carry out critical projects for the specialist department. The central Business Intelligence department does not only provide the technical delivery of data, but also through methodical consulting. Although most of the responsibility lies with the Business Intelligence department, some other data-focused departments are at least co-responsible.

The advantage is obvious: There are special data experts who work deeper in the actual departments and feel more connected and responsible to them. The technical-business focus lies on pain points of the company.

However, this form of Ogranization also has decisive disadvantages: The danger of developing isolated solutions that are so special in some specific areas that they will not really work company-wide increases. Typically the company has to deal with asymmetrical growth of data analytics
know-how. Managing data governance is more complex and recruitment is becoming more difficult as the business intelligence department is weakened and smaller, and data professionals for other departments need to have more business focus, which means they are looking for more specialized profiles.

Organization Type 3 – Decentral Approach

Some companies are also taking a more extreme approach in the other direction. The Business Intelligence department now has only Data Engineers building and maintaining the data warehouse or data lake. As a result, the central department only provides data; it is used and analyzed in all other departments, specifically for the respective applications.

The advantage lies in the personal responsibility of the respective departments as „pain points“ of the company are in focus in belief that business departments know their problems and solutions better than any other department does. Highly specialized data experts can understand colleagues of their own department well and there is no no shared service mindset neccessary, except for the data delivery.

Of course, this organizational form has clear disadvantages since many isolated solutions are unavoidable and the development process of each data-driven solution will be inefficient. These insular solutions may work with luck for your own department, but not for the whole company. There is no one single source of truth. The recruiting process is more difficult as it requires more specialized data experts with more business background. We have to expect an asymmetrical growth of data analytics know-how and a difficult data governance.

 

Amy Johnson

The 6 most in-demand AI jobs and how to get them

/in , , , /by

A press release issued in December 2017 by Gartner, Inc explicitly states, 2020 will be a pivotal year in Artificial Intelligence-related employment dynamics. It states AI will become “a positive job motivator”.

However, the Gartner report also sounds some alarm bells. “The number of jobs affected by AI will vary by industry-through 2019, healthcare, the public sector and education will see continuously growing job demand while manufacturing will be hit the hardest. Starting in 2020, AI-related job creation will cross into positive territory, reaching two million net-new jobs in 2025,” the press release adds.

This phenomenon is expected to strike worldwide, as a report carried by a leading Indian financial daily, The Hindu BusinessLine states. “The year 2018 will see a sharp increase in demand for professionals with skills in emerging technologies such as Artificial Intelligence (AI) and machine learning, even as people with capabilities in Big Data and Analytics will continue to be the most sought after by companies across sectors, say sources in the recruitment industry,” this news article says.

Before we proceed, let us understand what exactly does Artificial Intelligence or AI mean.

Understanding Artificial Intelligence

Encyclopedia Britannica explains AI as: “The ability of a digital computer or computer-controlled robot to perform tasks commonly associated with human beings.” Classic examples of AI are computer games that can be played solo on a computer. Of these, one can be a human while the other is the reasoning, analytical and other intellectual property a computer. Chess is one example of such a game. While playing Chess with a computer, AI will analyze your moves. It will predict and reason why you made them and respond accordingly.

Similarly, AI imitates functions of the human brain to a very great extent. Of course, AI can never match the prowess of humans but it can come fairly close.

What this means?

This means that AI technology will advance exponentially. The main objective for developing AI will not aim at reducing dependence on humans that can result in loss of jobs or mass retrenchment of employees. Having a large population of unemployed people is harmful to economy of any country. Secondly, people without money will not be able to utilize most functions that are performed through AI, which will render the technology useless.

The advent and growing popularity of AI can be summarized in words of Bill Gates. According to the founder of Microsoft, AI will have a positive impact on people’s lives. In an interview with Fox Business, he said, people would have more spare time that would eventually lead to happier life. However he cautions, it would be long before AI starts making any significant impact on our daily activities and jobs.

Career in AI

Since AI primarily aims at making human life better, several companies are testing the technology. Global online retailer Amazon is one amongst these. Banks and financial institutions, service providers and several other industries are expected to jump on the AI bandwagon in 2018 and coming years. Hence, this is the right time to aim for a career in AI. Currently, there exists a great demand for AI professionals. Here, we look at the top six employment opportunities in Artificial Intelligence.

Computer Vision Research Engineer

 A Computer Vision Research Engineer’s work includes research and analysis, developing software and tools, and computer vision technologies. The primary role of this job is to ensure customer experience that equals human interaction.

Business Intelligence Engineer

As the job designation implies, the role of a Business Intelligence Engineer is to gather data from multiple functions performed by AI such as marketing and collecting payments. It also involves studying consumer patterns and bridging gaps that AI leaves.

Data Scientist

A posting for Data Scientist on recruitment website Indeed describes Data Scientist in these words: “ A mixture between a statistician, scientist, machine learning expert and engineer: someone who has the passion for building and improving Internet-scale products informed by data. The ideal candidate understands human behavior and knows what to look for in the data.

Research and Development Engineer (AI)

Research & Development Engineers are needed to find ways and means to improve functions performed through Artificial Intelligence. They research voice and text chat conversations conducted by bots or robotic intelligence with real-life persons to ensure there are no glitches. They also develop better solutions to eliminate the gap between human and AI interactions.

Machine Learning Specialist

The job of a Machine Learning Specialist is rather complex. They are required to study patterns such as the large-scale use of data, uploads, common words used in any language and how it can be incorporated into AI functions as well as analyzing and improving existing techniques.

Researchers

Researchers in AI is perhaps the best-paid lot. They are required to research into various aspects of AI in any organization. Their role involves researching usage patterns, AI responses, data analysis, data mining and research, linguistic differences based on demographics and almost every human function that AI is expected to perform.

As with any other field, there are several other designations available in AI. However, these will depend upon your geographic location. The best way to find the demand for any AI job is to look for good recruitment or job posting sites, especially those specific to your region.

In conclusion

Since AI is a technology that is gathering momentum, it will be some years before there is a flood of people who can be hired as fresher or expert in this field. Consequently, the demand for AI professionals is rather high. Median salaries these jobs mentioned above range between US$ 100,000 to US$ 150,000 per year.

However, before leaping into AI, it is advisable to find out what other qualifications are required by employers. As with any job, some companies need AI experts that hold specific engineering degrees combined with additional qualifications in IT and a certificate that states you hold the required AI training. Despite, this is the best time to make a career in the AI sector.

Benjamin Aunkofer

My Desk for Data Science

/in , , , , /by

In my last post I anounced a blog parade about what a data scientist’s workplace might look like.

Here are some photos of my desk and my answers to the questions:

How many monitors do you use (or wish to have)?

I am mostly working at my desk in my office with a tower PC and three monitors.
I definitely need at least three monitors to work productively as a data scientist. Who does not know this: On the left monitor the data model is displayed, on the right monitor the data mapping and in the middle I do my work: programming the analysis scripts.

What hardware do you use? Apple? Dell? Lenovo? Others?

I am note an Apple guy. When I need to work mobile, I like to use ThinkPad notebooks. The ThinkPads are (in my experience) very robust and are therefore particularly good for mobile work. Besides, those notebooks look conservative and so I’m not sad if there comes a scratch on the notebook. However, I do not solve particularly challenging analysis tasks on a notebook, because I need my monitors for that.

Which OS do you use (or prefer)? MacOS, Linux, Windows? Virtual Machines?

As a data scientist, I have to be able to communicate well with my clients and they usually use Microsoft Windows as their operating system. I also use Windows as my main operating system. Of course, all our servers run on Linux Debian, but most of my tasks are done directly on Windows.
For some notebooks, I have set up a dual boot, because sometimes I need to start native Linux, for all other cases I work with virtual machines (Linux Ubuntu or Linux Mint).

What are your favorite databases, programming languages and tools?

I prefer the Microsoft SQL Server (T-SQL), C# and Python (pandas, numpy, scikit-learn). This is my world. But my customers are kings, therefore I am working with Postgre SQL, MongoDB, Neo4J, Tableau, Qlik Sense, Celonis and a lot more. I like to get used to new tools and technologies again and again. This is one of the benefits of being a data scientist.

Which data dou you analyze on your local hardware? Which in server clusters or clouds?

There have been few cases yet, where I analyzed really big data. In cases of analyzing big data we use horizontally scalable systems like Hadoop and Spark. But we also have customers analyzing middle-sized data (more than 10 TB but less than 100 TB) on one big server which is vertically scalable. Most of my customers just want to gather data to answer questions on not so big amounts of data. Everything less than 10TB we can do on a highend workstation.

If you use clouds, do you prefer Azure, AWS, Google oder others?

Microsoft Azure! I am used to tools provided by Microsoft and I think Azure is a well preconfigured cloud solution.

Where do you make your notes/memos/sketches. On paper or digital?

My calender is managed digital, because I just need to know everywhere what appointments I have. But my I prefer to wirte down my thoughts on paper and that´s why I have several paper-notebooks.

Now it is your turn: Join our Blog Parade!

So what does your workplace look like? Show your desk on your blog until 31/12/2017 and we will show a short introduction of your post here on the Data Science Blog!

 

Benjamin Aunkofer

Show your Data Science Workplace!

/in , , , /by

The job of a data scientist is often a mystery to outsiders. Of course, you do not really need much more than a medium-sized notebook to use data science methods for finding value in data. Nevertheless, data science workplaces can look so different and, let’s say, interesting. And that’s why I want to launch a blog parade – which I want to start with this article – where you as a Data Scientist or Data Engineer can show your workplace and explain what tools a data scientist in your opinion really needs.

I am very curious how many monitors you prefer, whether you use Apple, Dell, HP or Lenovo, MacOS, Linux or Windows, etc., etc. And of course, do you like a clean or messy desk?

What is a Blog Parade?

A blog parade is a call to blog owners to report on a specific topic. Everyone who participates in the blog parade, write on their blog a contribution to the topic. The organizer of the blog parade collects all the articles and will recap those articles in a short form together, of course with links to the articles.

How can I participate?

Write an article on your blog! Mention this blog parade here, show and explain your workplace (your desk with your technical equipment) in an article. If you’re missing your own blog, articles can also be posted directly to LinkedIn (LinkedIn has its own blogging feature that every LinkedIn member can use). Alternative – as a last resort – it would also be possible to send me your article with a photo about your workplace directly to: redaktion@data-science-blog.com.
Please make me aware of an article, via e-mail or with a comment (below) on this article.

Who can participate?

Any data scientist or anyone close to Data Science: Everyone concerned with topics such as data analytics, data engineering or data security. Please do not over-define data science here, but keep it in a nutshell, so that all professionals who manage and analyze data can join in with a clear conscience.

And yes, I will participate too. I will propably be the first who write an article about my workplace (I just need a new photo of my desk).

When does the article have to be finished?

By 31/12/2017, the article must have been published on your blog (or LinkedIn or wherever) and the release has to be reported to me.
But beware: Anyone who has previously written an article will also be linked earlier. After all, reporting on your article will take place immediately after I hear about it.
If you publish an artcile tomorrow, it will be shown the day after tomorrow here on the Data Science Blog.

What is in it for me to join?

Nothing! Except perhaps the fun factor of sharing your idea of ​​a nice desk for a data expert with others, so as to share creativity or a certain belief in what a data scientist needs.
Well and for bloggers: There is a great backlink from this data science blog for you 🙂

What should I write? What are the minimum requirements of content?

The article does not have to (but may be) particularly long. Anyway, here on this data science blog only a shortened version of your article will appear (with a link, of course).

Minimum requirments:

  • Show a photo (at least one!) of your workplace desk!
  • And tell us something about:
    • How many monitors do you use (or wish to have)?
    • What hardware do you use? Apple? Dell? Lenovo? Others?
    • Which OS do you use (or prefer)? MacOS, Linux, Windows? Virtual Machines?
    • What are your favorite databases, programming languages and tools? (e.g. Python, R, SAS, Postgre, Neo4J,…)
    • Which data dou you analyze on your local hardware? Which in server clusters or clouds?
    • If you use clouds, do you prefer Azure, AWS, Google oder others?
    • Where do you make your notes/memos/sketches. On paper or digital?

Not allowed:
Of course, please do not provide any information, which could endanger your company`s IT security.

Absolutly allowed:
Bringing some joke into the matter 🙂 We are happy to vote in the comments on the best or funniest desk for election, there may be also a winner later!

The resulting Blog Posts: https://data-science-blog.com/data-science-insights/show-your-desk/

 

Benjamin Aunkofer

Data Science vs Data Engineering

/in , , , , , , /by

The job of the Data Scientist is actually a fairly new trend, and yet other job titles are coming to us. “Is this really necessary?”, Some will ask. But the answer is clear: yes!

There are situations, every Data Scientist know: a recruiter calls, speaks about a great new challenge for a Data Scientist as you obviously claim on your LinkedIn profile, but in the discussion of the vacancy it quickly becomes clear that you have almost none of the required skills. This mismatch is mainly due to the fact that under the job of the Data Scientist all possible activity profiles, method and tool knowledge are summarized, which a single person can hardly learn in his life. Many open jobs, which are to be called under the name Data Science, describe rather the professional image of the Data Engineer.

Read this article in German:
“Data Science vs Data Engineering – Wo liegen die Unterschiede?“

What is a Data Engineer?

Data engineering is primarily about collecting or generating data, storing, historicalizing, processing, adapting and submitting data to subsequent instances. A Data Engineer, often also named as Big Data Engineer or Big Data Architect, models scalable database and data flow architectures, develops and improves the IT infrastructure on the hardware and software side, deals with topics such as IT Security , Data Security and Data Protection. A Data Engineer is, as required, a partial administrator of the IT systems and also a software developer, since he or she extends the software landscape with his own components. In addition to the tasks in the field of ETL / Data Warehousing, he also carries out analyzes, for example, to investigate data quality or user access. A Data Engineer mainly works with databases and data warehousing tools.

A Data Engineer is talented as an educated engineer or computer scientist and rather far away from the actual core business of the company. The Data Engineer’s career stages are usually something like:

  1. (Big) Data Architect
  2. BI Architect
  3. Senior Data Engineer
  4. Data Engineer

What makes a Data Scientist?

Although there may be many intersections with the Data Engineer’s field of activity, the Data Scientist can be distinguished by using his working time as much as possible to analyze the available data in an exploratory and targeted manner, to visualize the analysis results and to convert them into a red thread (storytelling). Unlike the Data Engineer, a data scientist rarely sees into a data center, because he picks up data via interfaces provided by the Data Engineer or provides by other resources.

A Data Scientist deals with mathematical models, works mainly with statistical procedures, and applies them to the data to generate knowledge. Common methods of Data Mining, Machine Learning and Predictive Modeling should be known to a Data Scientist. Data Scientists basically work close to the department and need appropriate expertise. Data Scientists use proprietary tools (e.g. Tools by IBM, SAS or Qlik) and program their own analyzes, for example, in Scala, Java, Python, Julia, or R. Using such programming languages and data science libraries (e.g. Mahout, MLlib, Scikit-Learn or TensorFlow) is often considered as advanced data science.

Data Scientists can have diverse academic backgrounds, some are computer scientists or engineers for electrical engineering, others are physicists or mathematicians, not a few have economical backgrounds. Common career levels could be:

  1. Chief Data Scientist
  2. Senior Data Scientist
  3. Data Scientist
  4. Data Analyst oder Junior Data Scientist

Data Scientist vs Data Analyst

I am often asked what the difference between a Data Scientist and a Data Analyst would be, or whether there would be a distinction criterion at all:

In my experience, the term Data Scientist stands for the new challenges for the classical concept of Data Analysts. A Data Analyst performs data analysis like a Data Scientist. More complex topics such as predictive analytics, machine learning or artificial intelligence are topics for a Data Scientist. In other words, a Data Scientist is a Data Analyst++ (one step above the Data Analyst).

And how about being a Business Analyst?

Business Analysts can (but need not) be Data Analysts. In any case, they have a very strong relationship with the core business of the company. Business Analytics is about analyzing business models and business successes. The analysis of business success is usually carried out by IT, and many business analysts are starting a career as Data Analyst now. Dashboards, KPIs and SQL are the tools of a good business analyst, but there might be a lot business analysts, who are just analysing business models by reading the newspaper…

Benjamin Aunkofer

Data Science Knowledge Stack – Abstraction of the Data Science Skillset

/in , , , , , , , , /by

What must a Data Scientist be able to do? Which skills does as Data Scientist need to have? This question has often been asked and frequently answered by several Data Science Experts. In fact, it is now quite clear what kind of problems a Data Scientist should be able to solve and which skills are necessary for that. I would like to try to bring this consensus into a visual graph: a layer model, similar to the OSI layer model (which any data scientist should know too, by the way).
I’m giving introductory seminars in Data Science for merchants and engineers and in those seminars I always start explaining what we need to work out together in theory and practice-oriented exercises. Against this background, I came up with the idea for this layer model. Because with my seminars the problem already starts: I am giving seminars for Data Science for Business Analytics with Python. So not for medical analyzes and not with R or Julia. So I do not give a general knowledge of Data Science, but a very specific direction.

A Data Scientist must deal with problems at different levels in any Data Science project, for example, the data access does not work as planned or the data has a different structure than expected. A Data Scientist can spend hours debating its own source code or learning the ropes of new DataScience packages for its chosen programming language. Also, the right algorithms for data evaluation must be selected, properly parameterized and tested, sometimes it turns out that the selected methods were not the optimal ones. Ultimately, we are not doing Data Science all day for fun, but for generating value for a department and a data scientist is also faced with special challenges at this level, at least a basic knowledge of the expertise of that department is a must have.

Read this article in German:
“Data Science Knowledge Stack – Was ein Data Scientist können muss“

Data Science Knowledge Stack

With the Data Science Knowledge Stack, I would like to provide a structured insight into the tasks and challenges a Data Scientist has to face. The layers of the stack also represent a bidirectional flow from top to bottom and from bottom to top, because Data Science as a discipline is also bidirectional: we try to answer questions with data, or we look at the potentials in the data to answer previously unsolicited questions.

The DataScience Knowledge Stack consists of six layers:

Database Technology Knowledge

A Data Scientist works with data which is rarely directly structured in a CSV file, but usually in one or more databases that are subject to their own rules. In particular, business data, for example from the ERP or CRM system, are available in relational databases, often from Microsoft, Oracle, SAP or an open source alternative. A good Data Scientist is not only familiar with Structured Query Language (SQL), but is also aware of the importance of relational linked data models, so he also knows the principle of data table normalization.

Other types of databases, so-called NoSQL databases (Not only SQL) are based on file formats, column or graph orientation, such as MongoDB, Cassandra or GraphDB. Some of these databases use their own programming languages ​​(for example JavaScript at MongoDB or the graph-oriented database Neo4J has its own language called Cypher). Some of these databases provide alternative access via SQL (such as Hive for Hadoop).

A data scientist has to cope with different database systems and has to master at least SQL – the quasi-standard for data processing.

Data Access & Transformation Knowledge

If data are given in a database, Data Scientists can perform simple (and not so simple) analyzes directly on the database. But how do we get the data into our special analysis tools? To do this, a Data Scientist must know how to export data from the database. For one-time actions, an export can be a CSV file, but which separators and text qualifiers should be used? Possibly, the export is too large, so the file must be split.
If there is a direct and synchronous data connection between the analysis tool and the database, interfaces like REST, ODBC or JDBC come into play. Sometimes a socket connection must also be established and the principle of a client-server architecture should be known. Synchronous and asynchronous encryption methods should also be familiar to a Data Scientist, as confidential data are often used, and a minimum level of security is most important for business applications.

Many datasets are not structured in a database but are so-called unstructured or semi-structured data from documents or from Internet sources. And again we have interfaces, a frequent entry point for Data Scientists is, for example, the Twitter API. Sometimes we want to stream data in near real-time, let it be machine data or social media messages. This can be quite demanding, so the data streaming is almost a discipline with which a Data Scientist can come into contact quickly.

Programming Language Knowledge

Programming languages ​​are tools for Data Scientists to process data and automate processing. Data Scientists are usually no real software developers and they do not have to worry about software security or economy. However, a certain basic knowledge about software architectures often helps because some Data Science programs can be going to be integrated into an IT landscape of the company. The understanding of object-oriented programming and the good knowledge of the syntax of the selected programming languages ​​are essential, especially since not every programming language is the most useful for all projects.

At the level of the programming language, there is already a lot of snares in the programming language that are based on the programming language itself, as each has its own faults and details determine whether an analysis is done correctly or incorrectly: for example, whether data objects are copied or linked as reference, or how NULL/NaN values ​​are treated.

Data Science Tool & Library Knowledge

Once a data scientist has loaded the data into his favorite tool, for example, one of IBM, SAS or an open source alternative such as Octave, the core work just began. However, these tools are not self-explanatory and therefore there is a wide range of certification options for various Data Science tools. Many (if not most) Data Scientists work mostly directly with a programming language, but this alone is not enough to effectively perform statistical data analysis or machine learning: We use Data Science libraries (packages) that provide data structures and methods as a groundwork and thus extend the programming language to a real Data Science toolset. Such a library, for example Scikit-Learn for Python, is a collection of methods implemented in the programming language. The use of such libraries, however, is intended to be learned and therefore requires familiarization and practical experience for reliable application.

When it comes to Big Data Analytics, the analysis of particularly large data, we enter the field of Distributed Computing. Tools (frameworks) such as Apache Hadoop, Apache Spark or Apache Flink allows us to process and analyze data in parallel on multiple servers. These tools also provide their own libraries for machine learning, such as Mahout, MLlib and FlinkML.

Data Science Method Knowledge

A Data Scientist is not simply an operator of tools, he uses the tools to apply his analysis methods to data he has selected for to reach the project targets. These analysis methods are, for example, descriptive statistics, estimation methods or hypothesis tests. Somewhat more mathematical are methods of machine learning for data mining, such as clustering or dimensional reduction, or more toward automated decision making through classification or regression.

Machine learning methods generally do not work immediately, they have to be improved using optimization methods like the gradient method. A Data Scientist must be able to detect under- and overfitting, and he must prove that the prediction results for the planned deployment are accurate enough.

Special applications require special knowledge, which applies, for example, to the fields of image recognition (Visual Computing) or the processing of human language (Natural Language Processiong). At this point, we open the door to deep learning.

Expertise

Data Science is not an end in itself, but a discipline that would like to answer questions from other expertise fields with data. For this reason, Data Science is very diverse. Business economists need data scientists to analyze financial transactions, for example, to identify fraud scenarios or to better understand customer needs, or to optimize supply chains. Natural scientists such as geologists, biologists or experimental physicists also use Data Science to make their observations with the aim of gaining knowledge. Engineers want to better understand the situation and relationships between machinery or vehicles, and medical professionals are interested in better diagnostics and medication for their patients.

In order to support a specific department with his / her knowledge of data, tools and analysis methods, every data scientist needs a minimum of the appropriate skills. Anyone who wants to make analyzes for buyers, engineers, natural scientists, physicians, lawyers or other interested parties must also be able to understand the people’s profession.

Engere Data Science Definition

While the Data Science pioneers have long established and highly specialized teams, smaller companies are still looking for the Data Science Allrounder, which can take over the full range of tasks from the access to the database to the implementation of the analytical application. However, companies with specialized data experts have long since distinguished Data Scientists, Data Engineers and Business Analysts. Therefore, the definition of Data Science and the delineation of the abilities that a data scientist should have, varies between a broader and a more narrow demarcation.


A closer look at the more narrow definition shows, that a Data Engineer takes over the data allocation, the Data Scientist loads it into his tools and runs the data analysis together with the colleagues from the department. According to this, a Data Scientist would need no knowledge of databases or APIs, neither an expertise would be necessary …

In my experience, DataScience is not that narrow, the task spectrum covers more than just the core area. This misunderstanding comes from Data Science courses and – for me – I should point to the overall picture of Data Science again and again. In courses and seminars, which want to teach Data Science as a discipline, the focus will of course be on the core area: programming, tools and methods from mathematics & statistics.

Benjamin Aunkofer

What makes a good Data Scientist? Answered by leading Data Officers!

/in , , , , /by

What makes a good Data Scientist? A question I got asked recently a lot by data science newbies as well as long-established CIOs and my answer ist probably not what you think:
In my opinion is a good Data Scientist somebody with, at least, a good knowledge of computer programming, statistics and the ability of understanding the customer´s business. Above all stands a strong interest in finding value in distributed data sources.

Debatable? Maybe. That’s why I forwarded this question to five other leading Data Scientists and Chief Data Officers in Germany, let’s have a look on their answers to this question and create your own idea of what a good Data Scientist might be:

Dr. Andreas Braun – Head of Global Data & Analytics @ Allianz SE

A data scientist connects thorough analytical and methodological understanding  with a technical hands-on/ engineering mentality.
Data scientists bridge between analytics, tech, and business. “New methods”, such as machine learning, AI, deep learning etc. are crucial and are continuously challenged and improved. (14 February 2017)

Dr. Helmut Linde – Head of Data Science @ SAP SE

The ideal data scientist is a thought leader who creates value from analytics, starting from a vision for improved business processes and an algorithmic concept, down to the technical realization in productive software. (09 February 2017)

Klaas Bollhoefer – Chief Data Scientist @ The unbelievable Machine Company

For me a data scientist thinks ahead, thinks about and thinks in-between. He/she is a motivated, open-minded, enthusiastic and unconventional problem solver and tinkerer. Being a team player and a lone wolf are two sides of the same coin and he/she definitely hates unicorns and nerd shirts. (27 March 2017)

 

Wolfgang Hauner – Chief Data Officer @ Munich Re

A data scientist is, from their very nature, interested in data and its underlying relationship and has the cognitive, methodical and technical skills to find these relationships, even in unstructured data. The essential prerequisites to achieve this are curiosity, a logical mind-set and a passion for learning, as well as an affinity for team interaction in the work place. (08 February 2017)

 

Dr. Florian Neukart – Principal Data Scientist @ Volkswagen Group of America

In my opinion, the most important trait seems to be driven by an irresistible urge to understand fundamental relations and things, whereby I summarize both an atom and a complex machine among “things”. People with this trait are usually persistent, can solve a new problem even with little practical experience, and strive for the necessary training or appropriate quantitative knowledge autodidactically. (08 February 2017)

Background idea:
That I am writing about atoms and complex machines has to do with the fact that I have been able to analyze the most varied data through my second job at the university, and that I am given a chance to making significant contributions to both machine learning and physics, is primarily rooted in curiosity. Mathematics, physics, neuroscience, computer science, etc. are the fundamentals that someone will acquire if she wants to understand. In the beginning, there is only curiosity… I hope this is not too out of the way, but I’ve done a lot of job interviews and worked with lots of smart people, and it has turned out that quantitative knowledge alone is not enough. If someone is not burning for understanding, she may be able to program a Convolutional Network from the ground but will not come up with new ideas.