Have you ever wondered where the innovative ideas come from? In our opinion, this is usually either a case of accident or a need to solve a problem. Deep Image, an application created by our full-stack developer - Andrzej, was created by accident to solve a certain problem concerning the quality of graphics files.
How can Deep Image help you?
Before we tell you what Deep Image is check if any of the following applies to you:
- you lost original graphics, and the only thing left is uploaded archives, in very poor quality, rather unprintable
- you have legal, archived graphics files of your favorite artists, you would like to have them in the form of a poster, but their resolution does not allow for printing
- you have an archive of private photos in low resolution and you would like to improve their quality
- you are interested in restoring old photographs from your family archive
- you need to enlarge or cut out a specific fragment of the photo/graphic (for example, school photography from years ago), but you are worried about the quality
- you work in CSI Miami and you have to enhance the picture from the city camera because in the passersby glasses you can see the criminal's face reflection
Deep Image will help you.
But how does it work?
Each graphic file is a matrix and a set of data stored in it (numbers - pixels). After enlarging the image, the amount of data does not increase, so the image obtained has visually worse quality. By default, filtering techniques e.g. bicubic interpolation (using Photoshop or other tools), are used to improve the value. Meanwhile, in Deep Image, thanks to the use of machine learning, we get a larger image with a much better quality compared to bicubic interpolation. By using the super-resolution (SR) technique the application reconstructs the image, or sequence with a higher resolution, from the low resolution (LR) images.
Neurons, modeled on human, in action
The core of the application is a CNN or ConvNet. In machine learning, it is a class of convolutional neural networks, which are successfully used for image analysis. CNNs are designed to require minimal pre-processing, compared to other image classification algorithms. It means that the network learns the filters that were manually developed in traditional algorithms.
The convolutional networks are based on biological processes, modeled on human neurons. In Deep Image networks are developing. The more examples they get (they analyze more data), the more intelligent the application will be. The network learns that the line on the graphic can not have sharp edges and should be smooth - more options for smoothing, the better the quality of output file is.
"We need to go deeper!"
Working with neural networks does not look like a standard programming process. More like science, based on showing the pattern. Check this:
- in Deep Image on the input we have low-resolution graphics, while on the output we have to get a high resolution. At the beginning of cooperation with the neural network, we set random parameters
- From the moment of entry, the neural network learns how to create good quality graphics with the help of various transformations
- The network counts an error by analyzing the difference between the input (the starting image) and the output (the final image). Then it modifies the weights so that the difference between successive exits is as small as possible.
- The learning and creation process is based on algorithms (filter sets). Neural network assimilates information about a given edge so that in the end the line is smooth.
Application operation is based on the iterative process, which brings the final graphics almost to perfection. Everything happens automatically through a framework that is used in Deep Image-Keras (high-level API written in Python for the tangled neural networks TensorFlow and Theano).
Remove artifacts / resize
The entire process uses 2 algorithms developed on the basis of scientific research:
Zoom algorithm (resize) is based on the scientific publication: "Accelerating the Super-Resolution Convolutional Neural Network", which describes the model with 7 layers of weave and one deconvolution layer. Each image was divided into 200x200 pixels and processed separately. The study describes the usage of stride - the number of pixels at which the convolution filter moves. Stride used in the reconstruction layer (at the end) allows to multiply the amount of outgoing data, eg. stride = 2 is a matrix 2 times larger, 3 to 3 times, etc., that is, what exactly we need when enlarging the image.
Jpg algorithm (remove artifacts) is based on the publication "Compression Artifacts Removal Using Convolutional Neural Networks". The model is not very complicated and consists of one convolution layer with 128 features with a 3x3 nucleus. Each image is divided into 200x200 pixels and processed separately.
Both algorithms can be compared to those found in 4K TVs - so-called upscaling system.
As you already know how Deep Image works, you can now check its operation in practice: CHECK HOW IT WORKS