Shannon Flynn

3 Types of Preventative Maintenance for Data Centers

/in /by

Image Source: source unsplash.com

Downtime for a data center can be extraordinarily costly — potentially leading to lost revenue, lost customers and a damaged reputation. Preventative maintenance (PM) helps keep essential data center equipment running for as long as possible (while also making potential issues easier to spot).

However, there are many strategies for preventative maintenance that a data center can use, and not every strategy will be right for every center.

These are 3 types of preventative maintenance that businesses can use to maximize data center uptime and extend the lifespan of key equipment.

What Is Preventative Maintenance?

With preventative maintenance, an asset owner performs regularly scheduled maintenance activities in order to prevent future failures, downtime or unplanned repairs. Regardless of industry, preventative maintenance tasks always have a few characteristics in common:

  1. The maintenance is systematic, meaning it is done according to a pre-established plan or method.
  2. The maintenance is regular, meaning it occurs at predetermined intervals.
  3. The maintenance is preventative, meaning that it is intended to prevent failures and unplanned repairs.

Any effective PM strategy requires coordination, documentation and scheduling. Managers will need to gather information on asset performance, develop a maintenance strategy and ensure that maintenance is being both properly performed and occurring at regular intervals.

Common examples of maintenance tasks in a data center include the physical inspection of servers, the review of server logs and software updates.

1. Time-Based/Calendar-Based Preventative Maintenance

Calendar-based maintenance occurs at a specific time, based on a calendar interval. For example, a data center may schedule a regular visual inspection of server vents to occur daily, weekly, or monthly. The same data center may also schedule bi-monthly backups of key digital assets.

Intervals are generally determined based on the maintenance task being performed and a combination of historical performance data and industry best practices.

A data center may determine its inspection schedule based on recommendations from business partners, experience with past failures and data on equipment performance that can show when equipment performance begins to degrade without maintenance or inspections.

These intervals will be a part of the data center’s overall maintenance plan and should be regularly reviewed to ensure that maintenance isn’t occurring too often or too infrequently.

Particularly intensive maintenance tasks — anything that requires a great deal of time, requires the disassembly or important equipment or requires that servers be taken offline — may need to be scheduled less frequently to balance the benefits of PM against the potential costs, like downtime.

2. Usage-Based Preventative Maintenance

With a usage-based PM strategy, maintenance tasks occur based on how frequently equipment is used. Instead of occurring automatically once enough time has passed, usage-based tasks only trigger when an asset has been online for long enough or experienced enough exposure to certain environmental conditions.

Usage-based PM is most useful for assets that are not used continuously. These assets may not degrade as quickly as assets that are used regularly or always online.

Some time-based maintenance may still be necessary for assets that otherwise benefit from usage-based maintenance. Components or equipment kept in storage can degrade over time due to environmental conditions like dust, UV or moisture. Inspecting these assets regularly can help businesses ensure that they are not degrading while not in use.

3. Predictive Maintenance (PdM)

A novel approach to improving preventative maintenance, predictive maintenance uses AI algorithms and big data analysis to forecast when maintenance will be necessary.

The algorithm uses historical asset performance data and real-time monitoring to see failure coming, allowing the asset owner to preemptively schedule maintenance in response to potential downtime. Common sources of real-time monitoring data include built-in equipment sensors, IoT monitoring devices and logging software.

Predictive maintenance can allow asset owners to minimize maintenance costs, reduce downtime and extend the lifespan of their assets.

Specific savings will vary from data center to data center, but the Department of Energy estimates that businesses can save between 8% to 12% on maintenance expenses by switching from PM to PdM. The same business would also cut downtime by 35% to 45%.

Using Preventative Maintenance in Data Centers

PM can be an invaluable tool for data center owners wanting to minimize downtime and maximize the lifespan of key assets.

Time-based PM or predictive maintenance will likely be most useful for assets that are online most of the time. Usage-based PM can be useful for assets that are used less frequently (or spend a great deal of time ideal or in storage).

Training of Deep Learning AI models
Benjamin Aunkofer

It’s All About Data: The Training of AI Models

/in , , , , /by

In deep learning, there are different training methods. Which one we use in an AI project depends on the data provided by our customer: how much data is there, is it labeled or unlabeled? Or is there both labeled and unlabeled data?

Let’s say our customer needs structured, labeled images for an online tourism portal. The task for our AI model is therefore to recognize whether a picture is a bedroom, bathroom, spa area, restaurant, etc. Let’s take a look at the possible training methods.

1. Supervised Learning

If our customer has a lot of images and they are all labeled, this is a rare stroke of luck. We can then apply supervised learning. The AI model learns the different image categories based on the labeled images. For this purpose, it receives the training data with the desired results from us.

During training, the model searches for patterns in the images that match the desired results, learning the characteristics of the categories. The model can then apply what it has learned to new, unseen data and in this way provide a prediction for unlabeled images, i.e., something like “bathroom 98%.”

2. Unsupervised Learning

If our customer can provide many images as training data, but all of them are not labeled, we have to resort to unsupervised learning. This means that we cannot tell the model what it should learn (the assignment to categories), but it must find regularities in the data itself.

Contrastive learning is currently a common method of unsupervised learning. Here, we generate several sections from one image at a time. The model should learn that the sections of the same image are more similar to each other than to those of other images. Or in short, the model learns to distinguish between similar and dissimilar images.

Although we can use this method to make predictions, they can never achieve the quality of results of supervised learning.

3. Semi-supervised Learning

If our customer can provide us with few labeled data and a large amount of unlabeled data, we apply semi-supervised learning. In practice, we actually encounter this data situation most often.

With semi-supervised learning, we can use both data sets for training, the labeled and the unlabeled data. This is possible by combining contrastive learning and supervised learning, for example: we train an AI model with the labeled data to obtain predictions for room categories. At the same time, we let the model learn similarities and dissimilarities in the unlabeled data and then optimize itself. In this way, we can ultimately achieve good label predictions for new, unseen images.

Supervised vs. Unsupervised vs. Semi-supervised

Everyone who is entrusted with an AI project wants to apply supervised learning. In practice, however, this is rarely the case, as rarely all training data is well structured and labeled.

If only unstructured and unlabeled data is available, we can at least extract information from the data with unsupervised learning. These can already provide added value for our customer. However, compared to supervised learning, the quality of the results is significantly worse.

With semi-supervised learning, we try to resolve the data dilemma of small part labeled data, large part unlabeled data. We use both datasets and can obtain good prediction results whose quality is often on par with those of supervised learning. This article is written in cooperation between DATANOMIQ and pixolution, a company for computer vision and AI-bases visual search.

Jieyun Hu

Automatic Financial Trading Agent for Low-risk Portfolio Management using Deep Reinforcement Learning

/in , , , , /by

This article focuses on autonomous trading agent to solve the capital market portfolio management problem. Researchers aim to achieve higher portfolio return while preferring lower-risk actions. It uses deep reinforcement learning Deep Q-Network (DQN) to train the agent. The main contribution of their work is the proposed target policy.

Introduction

Author emphasizes the importance of low-risk actions for two reasons: 1) the weak positive correlation between risk and profit suggests high returns can be obtained with low-risk actions, and 2) customer satisfaction decreases with increases in investment risk, which is undesirable. Author challenges the limitation of Supervised Learning algorithm since it requires domain knowledge. Thus, they propose Reinforcement Learning to be more suitable, because it only requires state, action and reward specifications.

The study verifies the method through the back-test in the cryptocurrency market because it is extremely volatile and offers enormous and diverse data. Agents then learn with shorter periods and are tested for the same period to verify the robustness of the method. 

2 Proposed Method

The overall structure of the proposed method is shown below.

The architecutre of the proposed trading agent system.

The architecutre of the proposed trading agent system.

2.1 Problem Definition

The portfolio consists of m assets and one base currency.

The price vector p stores the price p of all assets:

The portfolio vector w stores the amount of each asset:

At time 𝑡, the total value W_t of the portfolio is defined as the inner product of the price vector p_t and the portfolio vector w_t .

Finally, the goal is to maximize the profit P_t at the terminal time step 𝑇.

2.2 Asset Data Preprocessing

1) Asset Selection
Data is drawn from the Binance Exchange API, where top m traded coins are selected as assets.

2) Data Collection
Each coin has 9 properties, shown in Table.1, so each trade history matrix has size (α * 9), where α is the size of the target period converted into minutes.

3) Zero-Padding
Pad all other coins to match the matrix size of the longest coin. (Coins have different listing days)

Comment: Author pointed out that zero-padding may be lacking, but empirical results still confirm their method covering the missing data well.

4) Stack Matrices
Stack m matrices of size (α * 9) to form a block of size (m* α * 9). Then, use sliding window method with widow size w to create (α – w + 1) number of sequential blocks with size (w *  m * 9).

5) Normalization
Normalize blocks with min-max normalization method. They are called history block 𝜙 and used as input (ie. state) for the agent.

3. Deep Q-Network

The proposed RL-based trading system follows the DQN structure.

Deep Q-Network has 2 networks, Q- and Target network, and a component called experience replay. The Q-network is the agent that is trained to produce the optimal state-action value (aka. q-value).

Comment: Q-value is calculated by the Bellman equation, which, in short, consists of the immediate reward from next action, and the discounted value of the next state by following the policy for all subsequent steps.

 

Here,
Agent: Portfolio manager
Action a: Trading strategy according to the current state
State 𝜙 : State of the capital market environment
Environment: Has all trade histories for assets, return reward r and provide next state 𝜙’ to agent again

DQN workflow:

DQN gets trained in multiple time steps of multiple episodes. Let’s look at the workflow of one episode.

Training of a Deep Q-Network

Training of a Deep Q-Network

1) Experience replay selects an action according to the behavior policy, executes in the environment, returns the reward and next state. This experience set (\phi_t, a_t, r_r,\phi_{t+!}) is stored in the repository as a sample of training data.

2) From the repository of prior observations, take a random batch of samples as the input to both Q- and Target network. The Q-network takes the current state and action from each data sample and predicts the q-value for that particular action. This is the ‘Predicted Q-Value’.Comment: Author uses 𝜀-greedy algorithm to calculate q-value and select action. To simplify, 𝜀-greedy policy takes the optimal action if a randomly generated number is greater than 𝜀, which represents a tradeoff between exploration and exploitation.

The Target network takes the next state from each data sample and predicts the best q-value out of all actions that can be taken from that state. This is the ‘Target Q-Value’.

Comment: Author proposes a different target policy to calculate the target q-value.

3) The Predicted q-value, Target q-value, and the observed reward from the data sample is used to compute the Loss to train the Q-network.

Comment: Target Network is not trained. It is held constant to serve as a stable target for learning and will be updated with a frequency different from the Q-network.

4) Copy Q-network weights to Target network after n time steps and continue to next time step until this episode is finished.

The architecutre of the proposed trading agent system.

4.0 Main Contribution of the Research

4.1 Action and Reward

Agent determines not only action a but ratio , at which the action is applied.

  1. Action:
    Hold, buy and sell. Buy and sell are defined discretely for each asset. Hold holds all assets. Therefore, there are (2m + 1) actions in the action set A.

    Agent obtains q-value of each action through q-network and selects action by using 𝜀-greedy algorithm as behavior policy.
  2. Ratio:
    \sigma is defined as the softmax value for the q-value of each action (ie. i-th asset at \sigma = 0.5 , then i-th asset is bought using 50% of base currency).
  3. Reward:
    Reward depends on the portfolio value before and after the trading strategy. It is clipped to [-1,1] to avoid overfitting.

4.2 Proposed Target Policy

Author sets the target based on the expected SARSA algorithm with some modification.

Comment: Author claims that greedy policy ignores the risks that may arise from exploring other outcomes other than the optimal one, which is fatal for domains where safe actions are preferred (ie. capital market).

The proposed policy uses softmax algorithm adjusted with greediness according to the temperature term 𝜏. However, softmax value is very sensitive to the differences in optimal q-value of states. To stabilize  learning, and thus to get similar greediness in all states, author redefine 𝜏 as the mean of absolute values for all q-values in each state multiplied by a hyperparameter 𝜏’.

4.3 Q-Network Structure

This study uses Convolutional Neural Network (CNN) to construct the networks. Detailed structure of the networks is shown in Table 2.

Comment: CNN is a deep neural network method that hierarchically extracts local features through a weighted filter. More details see: https://towardsdatascience.com/stock-market-action-prediction-with-convnet-8689238feae3.

5 Experiment and Hyperparameter Tuning

5.1 Experiment Setting

Data is collected from August 2017 to March 2018 when the price fluctuates extensively.

Three evaluation metrics are used to compare the performance of the trading agent.

  • Profit P_t introduced in 2.1.
  • Sharpe Ratio: A measure of return, taking risk into account.

    Comment: p_t is the standard deviation of the expected return and P_f  is the return of a risk-free asset, which is set to 0 here.
  • Maximum Drawdown: Maximum loss from a peak to a through, taking downside risk into account.

5.2 Hyperparameter Optimization

The proposed method has a number of hyperparameters: window size mentioned in 2.2,  𝜏’ in the target policy, and hyperparameters used in DQN structure. Author believes the former two are key determinants for the study and performs GridSearch to set w = 30, 𝜏’ = 0.25. The other hyperparameters are determined using heuristic search. Specifications of all hyperparameters are summarized in the last page.

Comment: Heuristic is a type of search that looks for a good solution, not necessarily a perfect one, out of the available options.

5.3 Performance Evaluation

Benchmark algorithms:

UBAH (Uniform buy and hold): Invest in all assets and hold until the end.
UCRP (Uniform Constant Rebalanced Portfolio): Rebalance portfolio uniformly for every trading period.

Methods from other studies: hyperparameters as suggested in the studies
EG (Exponential Gradient)
PAMR (Passive Aggressive Mean Reversion Strategy)

Comment: DQN basic uses greedy policy as the target policy.

The proposed DQN method exhibits the best overall results out of the 6 methods. When the agent is trained with shorter periods, although MDD increases significantly, it still performs better than benchmarks and proves its robustness.

6 Conclusion

The proposed method performs well compared to other methods, but there is a main drawback. The encoding method lacked a theoretical basis to successfully encode the information in the capital market, and this opaqueness is a rooted problem for deep learning. Second, the study focuses on its target policy, while there remains room for improvement with its neural network structure.

Specification of Hyperparameters

Specification of Hyperparameters.

 

References

  1. Shin, S. Bu and S. Cho, “Automatic Financial Trading Agent for Low-risk Portfolio Management using Deep Reinforcement Learning”, https://arxiv.org/pdf/1909.03278.pdf
  2. Li, P. Zhao, S. C. Hoi, and V. Gopalkrishnan, “PAMR: passive aggressive mean reversion strategy for portfolio selection,” Machine learning, vol. 87, pp. 221-258, 2012.
  3. P. Helmbold, R. E. Schapire, Y. Singer, and M. K. Warmuth, “On‐line portfolio selection using multiplicative updates,” Mathematical Finance, vol. 8, pp. 325-347, 1998.

https://deepai.org/machine-learning-glossary-and-terms/softmax-layer#:~:text=The%20softmax%20function%20is%20a,can%20be%20interpreted%20as%20probabilities.

http://www.kasimte.com/2020/02/14/how-does-temperature-affect-softmax-in-machine-learning.html

https://towardsdatascience.com/reinforcement-learning-made-simple-part-2-solution-approaches-7e37cbf2334e

https://towardsdatascience.com/reinforcement-learning-explained-visually-part-4-q-learning-step-by-step-b65efb731d3e

https://towardsdatascience.com/reinforcement-learning-explained-visually-part-3-model-free-solutions-step-by-step-c4bbb2b72dcf

https://towardsdatascience.com/reinforcement-learning-explained-visually-part-5-deep-q-networks-step-by-step-5a5317197f4b

Nadica Metuleva

How Online Businesses Can Mitigate Fraud Risk

/in /by

Fraud has the potential to shatter businesses of all sizes and in all industries. Now that many businesses operate online at least partially, if not completely, the fraud risks are more prominent than ever. Right alongside the perks of reaching an enormous audience and using endless marketing tricks for promotion, businesses have to find a way to mitigate such risks.

One global economic crime survey, from PwC, found that 47% of all businesses worldwide experienced some type of fraud in the last 2 years. While online sales are higher than ever and are expected to grow significantly, this is all matched by a growth in fraud.

If we stop to take a look at how the eCommerce world has progressed in just a few years, the risks are becoming more imminent. Nowadays, it is more important than ever to take action to mitigate risks.

These days, online retailers deal with approximately 206,000 attacks on their businesses each month, research shows. Cybercriminals keep looking – and finding – new ways to capture and use data obtained from businesses, something that can ruin the brand entirely.

If you operate your business online, it is your obligation to your customers and your company to protect if from fraudsters that will steal data and wreck your online reputation. A single instance of fraud can alienate many of your customers and damage your brand, often without any chance to fix it.

Your job is to continuously track the trends, know the risks, and practice data science security hacks to mitigate fraud risks. In this article, you’ll learn all about it. But first, let’s take a look at why fraud happens in the first place.

Why does online fraud take place?

There are two big reasons why fraudsters can get access to data on your website and ruin your business:

  • It is easy. Before the Internet existed and businesses were solely physical, fraudsters needed to do things like rob the place or steal physical cards to make purchases with. These days, fraudsters use their hacking skills to buy cards and make purchases, get access to customer data on your website, etc.
  • It’s often conducted anonymously. Scamming online stores gives fraudsters a high sense of anonymity. They cannot be caught on camera and they can operate from any location in the world. Most police departments don’t make this a priority, so most of them remain uncaught, while businesses suffer the consequences.

Unless you take precautions to prevent this from happening, you are opening your company to many fraud risks. The good thing is, you can actually take precautions and measures to prevent and minimize the effects of fraud when it happens.

How to mitigate fraud risks for your online business

Now that you know how frequently this happens – and why that is the case, it’s time to go through some actionable tips on how to minimize the risks.

1.    Use quality tools for modern fraud monitoring

Did you know that you can use tools to monitor and prevent fraud? Modern tools that are rich with features can protect your business’ data, as well as protect it from risky transactions. If you take a look at this guide on modern fraud monitoring, from SEON, a top-rated tool used for this purpose, you’ll find that there’s a lot to be done to mitigate such risks.

Some of the key features to benefit from when it comes to such tools are:

  • Real-time monitoring – at all times
  • Behavior tracking
  • Fraud scoring
  • Graph visualization
  • Risk-based authentication
  • Manual queries
  • Alerts and reporting
  • Sandboxing capacity

Thankfully, SEON has all that and more. Thanks to SEON, businesses can now authenticate their customers, automatically cancel or detect risky orders, block visitors based on geolocation, and create a variety of custom filters based on their preferences.

Source: seon.io

2.    Know your fraud risks

It’s impossible to prevent something that you don’t know anything about. Many companies aren’t even aware of the risks before they actually happen. When they realize it, the damage is already done.

Let’s go through the main types of fraud risks that you should work to mitigate today:

  • Credit card fraud

This type of fraud is a banking data crime. It’s a big term that includes all sorts of stealing and illegally using credit card information. In some cases, criminals will use stolen credit card information to buy services or products on your website.

In more severe cases, they’ll be able to get this from your website, which means that you aren’t keeping your customer’s payment details safe enough.

Either way, you are looking at grand losses and problems. Eventually, when people use stolen cards, this defrauds the business owners that have to refund the purchase.

  • Chargeback fraud

Chargeback fraud happens when a credit card provider asks the retailer to refund a disputed or fraudulent transaction. This happens when people buy a product or a service, receive it, but then request a full refund from the company that provided them with the card.  It is also known as friendly fraud. In most cases, criminals wait a few weeks or even a few months after receiving the goods, and then contact the bank to dispute a transaction ‘they don’t know happened’. Some merchants are too busy to notice this, so they are losing tons of money because of it.

  • Affiliate fraud

Affiliate fraud is done when criminals use fake data to generate affiliate commissions. In the affiliate marketing world, online businesses pay affiliates commission for clicks or sales they refer to the website. Criminals often game these systems and make it seem like there’s real activity to generate commissions or increase their amount.

  • Phishing schemes

This is one of the gravest and yet, most common frauds for online businesses. Most online businesses today provide their customers with accounts to facilitate their purchasing process and track their behaviors. This is where financial data, personal information, and purchase history are all stored. Through phishing schemes, fraudsters obtain this personal data, log into the accounts, and make unauthorized purchases.

These are just a few types of eCommerce fraud that occurs online. If you want to prevent them, you need to learn what your business is at risk for, and use the necessary tools to mitigate those risks.

3.    Audit your website regularly

Your website is your storefront and it is one of the most important things to work on. You shouldn’t just work on its design or the content you publish on it. If you want to discover flaws in it before fraudsters do and use it to their benefit, you need to audit it carefully – and regularly.

Using fraud detection tools is a great step toward this, but you should also make sure to check some other things, too.

For example, are your shopping cart plugins and software up-to-date?

Do you have a working SSL certificate or is it expired?

Does your site comply with the current data protection laws and regulations?

Is your store Payment Card Industry Data Security Standard (PCI-DSS) compliant?

Do you back it up as often as you should?

Have you updated your passwords recently, your hosting dashboard, and your CMS database?

4.    Pay close attention to high-value orders

Small frauds can cost you a bit of money and a bit of your reputation. Big frauds can kill your business and your reputation in the industry. This is why you should pay close attention to high-value orders before shipping them out.

Check these personally, even the gift cards. Such items are very often used by fraudsters who hope to resell them, but have obtained them illegally.

5.    Don’t be afraid to contact your customers

Customers that buy from you regularly will have similar behaviors every time they make a purchase. Your system will start flagging any unexpected behavior on their behalf. When that happens and you notice that an existing customer changed their patterns dramatically, don’t be afraid to reach out to them. This might save them and you a lot of money and keep them safer. Not to mention, it will make your brand even more trustworthy and secure in their eyes.

6.    Request the CVV number for purchases

The back of cards such as Visa, MasterCard, and Discover contains a three-digit security code called the Card Verification Value or CVV. American Express cards have a four-digit code on the back.

Why is it smart to request this number?

Most fraudsters have the card numbers and expiry date but don’t have the CVV. This will minimize the risks and make it impossible for them to make fraudulent purchases if they don’t have the physical card on them.

7.    Limit the amount of customer data you are collecting

It can be tempting to collect tons of customer data, especially for research. You can use this data to improve your marketing strategies and your brand and offer customers a more personalized experience. But, collecting a lot of data means that you are creating more risks for that data to be stolen.

That being said, make it your mission to collect and store as little data as possible. Collect only what is necessary.

Are you already doing these things?

Fraudsters are getting smarter about how they attack online businesses. It is your obligation to keep up with the scams in the digital world and find ways to mitigate the risks. This article gives you seven excellent starting points for this.