In late May in 2020 researchers from Carnegie Mellon University reported that of the 200 million comments that discussed the Coronavirus on Twitter, 45% were generated by machines. How this has affected the public debate and policymakers is difficult to determine but the potential risk of the spread of misinformation and more still stands. How “intelligent” are these powerful and equally valuable but possibly harmful machines? How are they built?

In the below, we will dive into the concept of Language Models the cornerstone of Natural Language Processing (NLP), what they are, how they are built, and how they can…


https://codeburst.io/what-is-regularization-in-machine-learning-aed5a1c36590

When training your deep learning models overfitting and underfitting are common problems that data scientists have to deal with. However in particular for the issue of overfitting, there are various methods to allow your model to generalize and become more robust. For that, data augmentation methods in order to enhance your given dataset to be more rich and varied can be rather handy. But what if the issue cannot solely be solved by optimizing your dataset, but only by allowing your model to generalize better when training. That is when we utilize “Regularization” methods.

What is Regularization?

Regularization may be defined as any…


The power of deep Learning

Congratulations, you have built your own deep learning model. You have trained the various layers with defined weights and your model has performed relatively well on a validation set. Nevertheless, your error rate is still fairly high and does not live up to your expectations. How can you further improve your model, for it to be reliably used in inference?

Model Improvement Methods:

  1. Learning Rate Finder
  2. Unfreezing and transfer learning + Discriminative Learning Rates
  3. Deeper Architectures
  4. More and more…

The Learning Rate Finder

Choosing the learning rate can be a double-edged sword. A learning rate can either be too low which results in an…


From left: Robin Li (Baidu), Jack Ma (Alibaba) and Pony Ma (Tencent)

Starting a business is tough. Starting a business in China is even tougher. Having built my own startup in China over the last two years I scratched the surface on how brutal the Chinese entrepreneurial landscape is, how it affects businesses to operate, build products, and compete and what kind of mindset Chinese entrepreneurs have.

While around 90% of western startups fail and only 1 out of 10 turns into a prosperous and sustainable business the numbers look even grimmer in China. Looking at data over the last few years, during which venture capital investments in China hit a new…


Overfitting a model is one of the biggest fears machine learning experts have when training there model. The trickiness is that if a model has been overfitted, one will only really find out once it is being used on data outside the training set. Overfitting a model means that the model is super good at making correct predictions, but only then when making predictions based on the training data. The model is useless once it is working with actual real data outside the training set. All the weights have been highly personalized to only fit the training set.

Overfitting happens…


A fresh machine-learning student who is diving deep into deep learning and attempting to build his own models is quick to realize, that the “state-of-the-art” is far from very perfect. Many generally accepted methods are somewhat inefficient and produce a result that can easily be toppled if one is willing to accept that this field is very young and ready to be newly discovered every day. Every day new discoveries are made which push new models to further increase accuracy. Some of the discoveries are very profound, but others are smart and fresh approaches to common problems. …


Accruing a large amount of data is relatively simple. Data can be scraped, created or copied and then be stored in huge data storages.

A key driver in developing an intelligent model, however, is not just a sheer mass of data but also an effective strategy to intelligently label data to add structure and sense to the data. Data labeling can, therefore, be described as a way to organize information depending on its content.

This content determines the tag or label to be assigned to a specific piece of information after it has been processed. …


A neural network and its layers

Activation functions are an extremely important feature of artificial neural networks. They basically decide whether a neuron should be activated or not. What, however, does it mean for a neuron to be activated and what role does it play in the neural network?

Neural networks have neurons that work in correspondence of weight, bias and their respective activation function. In a neural network, we would update the weights and biases of the neurons on the basis of the error at the output.

Moreover, a neural network consists of 3 types of different layers. The Input Layer accepts input features. It…


Loss Functions in Machine Learning

Machine Learning is all about building a trained model that receives input data and correctly predict results. How, however, can a computer, be able to “learn” and be trained? In its very essence, a computer utilizes an algorithm to minimize the error of its prediction and the actual observations. After every prediction, no matter if correct or incorrect he inches closer and closer by attempting to minimize its own prediction error. The result is a trained model that is not only able to predict the correct outcome of the dataset it was trained on, but also that of a so-called…


Lesson 8 of the FastAI took a different approach to teach and understand deep learning than Lesson 1–7 did. While Lesson 1–7 prioritizes a top-bottom approach, essentially enabling its student to build applications right away, Lesson 8 started with a bottom-up approach. The idea is to start from the very basics and make student build their own model starting with a basic Convolutional Neural Network (CNN) model. During lesson 8 I was especially fascinated by the introduced Kaiming Initialization and the respective paper. While I was excited to base my first paper on the Kaiming Initialization, I quickly realized that…

John Kaller

Global Tech Entrepreneur / Co-Founder @ unpackAI

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store