Are you interested in movies or TV series? Why not? You, me and almost everyone in this biosphere is interested. The wizarding world of Harry Potter, the seven kingdoms of Game of Thrones, Marvel Cinematic Universe, Ice Age, Lord of the Rings’ Middle Earth and each and every other fictional multiverse do not just make us wonder, but also make us want to live inside them. Not only the movies, but also some unforgettable characters like Gollum in Lord of the Rings, Dinosaurs in Jurassic Park, King Kong and Godzilla, as well as scenes like Iron Man’s suit-up, Bruce Banner transforming into Hulk and the ‘iconic CGI fight between Juggernaut and Colossus’ in Deadpool, succeeded in leaving an arresting impression in our hearts.
Have you ever thought how the filmmakers make this whole other fantasy world a reality? Get on board, let’s find out. You’re reading this article on this magazine which was initially created on a monitor — a display that has two real dimensions, height and width. Text, pictures and vectors are all on a single plane simply within x and y coordinates. When you are watching an animation movie like “Ice Age”, you look into the same single plane with height and width and you perceive a third dimension: Depth. How does your computer trick your eyes into think that the flat screen extends deep into an entire enormous universe?
The Third Dimension
Let’s imagine that this paper is a cartesian plane, 100 units wide and 100 units tall, with the origin in the upper left corner; the zero-zero point. We are going to draw a shape using the following commands. First, we reset, which clears the entire paper, and sets the brightness of lines to zero. Then we move the drawing point down to 50 50 which is the center and set the line intensity to 100%. With the intensity up, now we move to 100, 50, then 60, 75 and then back to 50,50. The last thing to do is set our line intensity back to 0%. Cool! We got a triangle! In 3D graphics, points have not just two coordinates, but three — X, Y and Z. Of course, we don’t have X/Y/Z coordinates on a 2D computer screen. So how we are going to draw these 3 coordinates on two? There comes Graphic Algorithms as a solution! They are responsible for “flattening” 3D coordinates onto a 2D plane. This process is known as 3D Projection. Imagine building a cube out of chopsticks and shining a flash light on it. The shadow it casts onto your wall, its projection– is flat. If you rotate the cube around, you can see that it’s a 3D object, even though it’s a flat projection. This transformation from 3D to 2D is exactly what your computer is doing, just with a lot more mathematics and less chopsticks.
Look at this beautiful teapot made from large number of triangles forming polygons which eventually forms a mesh. The denser the mesh, the smoother the curves and the finer the details. Why triangles? The reason triangles are used, and not squares, or polygons, or some other more complex shape is simplicity: three points in space unambiguously define a plane. Into the bargain, two points aren’t enough to define a plane, only a line. Therefore, three is the minimal and the perfect number which defines a plane. Triangles for the win! Mathematical functions define points in space forming triangles, which forms a polygon, finally a mesh. Ultimately, mathematical functions create the shapes. For vision, we require three things: the eye, the light and a thing to see. Now we have got a bunch of mathematical functions used to create a teapot; technically a thing to see. In computer graphical term the EYE becomes the camera. And this same camera scene is the 3D to 2D projection. The classic algorithm for doing this 3D to 2D projection is called Scanline Rendering, first developed in 1967 at the University of Utah. For instance, let’s consider one polygon. The goal is to figure out how this polygon translates to fill pixels on a computer screen, so let’s first overlay a grid of pixels to fill. The scanline algorithm starts by reading the three points that make up the polygon and finding the lowest and highest Y values. It will only consider rows between these two points.
Then, the algorithm works down one row at a time. In each row, it calculates where a line – running through the center of a row– intersects with the side of the polygon. Since polygons are triangles, if you intersect one line, you must intersect with another. It’s guaranteed! The algorithm, subsequently colors in all pixels between those two intersections. And this just continues, row by row, which is why it’s called Scanline Rendering. When we hit the bottom of the polygon, it’s good to go! Admittedly, this is seemingly a pretty bad-looking polygon. It has those rough edges which are known to be the “Jaggies”. One method to soften this effect is “Antialiasing”. Instead of filling pixels in a polygon with the same color, we can adjust the color based on how much the polygon cuts through each pixel. If a pixel is entirely inside of a polygon, it gets fully colored. But if the polygon only grazes a pixel, it’ll get a lighter shade. This feathering of the edges is much more pleasant to the eyes. In a 3D scene, there are polygons that are part objects in the back, near the front, and just about everywhere. Only some are visible, because some objects are hidden behind other objects in the scene which are called occlusion. The most straightforward way to handle this, is to use a sort algorithm and arrange all the polygons in the scene from farthest to nearest, then render them in that order. This is called the Painter’s Algorithm, because painters also must start with the background, and then increasingly work up to foreground elements.
It is all about lighting. To make your brain to believe what you are looking at is plain text in a magazine, is all depending on the magazine’s shadows, reflection, refraction, and all other lighting conditions. In the same way, for the computer graphic to be photorealistic, the most successful way would be the simulation of lights behavior on the graphic itself. In 1980, an engineer working at Bell Laboratories named Turner Whitted released a paper entitled “An Improved Illumination Model for Shaded Display” where he solved the shadow, reflection and refraction simulation problems single-handed. This was called recursive ray tracing. Starting from the camera, there are millions of rays going to the scenes’ objects. These are called primary rays. Nevertheless, when the primary ray contacts a surface, Whitted has us drawing secondary rays. In order to solve the shadow issue, we draw a shadow ray by drawing a secondary ray in the direction of the lights in the object space. If we find that there are no objects between the light and the surface, then we know that the light is directly illuminating the surface. Therefore, when we go to color that pixel, we will include specular and highlight values created by the light. If we find there is an object that is between the light and the surface, then we know that the surface is in shadow and we shade that pixel using only the ambient light value. Now if that surface is reflective, we draw a reflection ray using the angle of incidence and see where this reflection ray lands.
The information from this reflection ray will change how we shade this pixel. If the reflection ray lands on an object – we must draw new secondary rays from this new intersection recursively which is why this is called recursive ray tracing. A similar process is needed if the object is transparent. Instead using the angle of incidence, we use the index of refraction to determine the angle new refractive rays that we must draw. This was one of Turner Whitted’s very first ray traced images from his paper in 1980.In this image, we see shadows, reflections and refractions. This 512×512 image took 74 minutes to render. The cost of trying to get closer and closer to simulating reality of light is computational power.Even though Pixar’s first fulllength fully CGI Film: Toy Story came out in 1995, it wasn’t until a decade later with Cars released in 2005 when Pixar fully implemented ray tracing in their rendering Pipeline. Whitted’s paper signaled a fundamental shift in computer graphics research.
Shift in Computational Power
Central Processing Units were designed to perform the math alongside with the logic, resulting slower performance in furnishing visuals. Therefore, computer engineers came up with special processors just for graphics – a GPU, or Graphics Processing Unit. These can be found on graphics cards inside of your computer, along with RAM reserved for graphics. This is where all the meshes and textureslive, allowing them to be accessed super-fast by many different cores of the GPU all at once. A modern graphics card like a GeForce GTX 1080 TI, contains 3584 processing cores, offering massive parallelization. It can process hundreds of millions of polygons every second! What took Turner Whitted 74 minutes to render in 1980 can be done in a 1/30th of a second with today’s real time Ray Tracing thanks in part to Moore’s Law – an observation that the number of transistors in an integrated circuit doubles every two years. However, to balance out Moore’s Law there’s something called Blinn’s Law which states: As technology advances, rendering time remains constant. The more our machines are capable of, the more we throw at them.
Computer Generated Imagery
At heart – all movie is a special effect. From the earliest silent shorts to today’s digital creations, it’s just a bunch of 2D images flashed in quick succession to rick the brain into seeing motion. However, the filmmakers, just like magicians, have been tremendously working on that splendid magic trick. Introducing cutting, matte paintings, compositing, miniatures, puppets and now photo realistic Computer-Generated imagery. We’ve gotten so used to the magic trick that these amazing models of optics and light physics needed to create stunning CGI filmmaking techniques, but how can you look at the monumental task of getting a calculator to produce an image and not be amazed by all mathematics, engineering and computer science that has been accomplished in less than half a century? More importantly, these tools enable all of us to do what was once only in the realm of a very few well equipped and well-funded individuals. CGI is a tool, an amazing tool while every other visual artform involves the manipulation of some natural phenomenon but CGI is born completely from the ground up out of our human imagination.
We said, here’s how we want it to work and we made it. How can you not be amazed by the potential of human ingenuity? And when we employ that power in movies to enhance and support the story, it can open up whole new worlds of possibility for the narrative. All that matters are what’s on the screen – how the magic trick is done is really nothing more than a piece of trivia. Go out there and make something great. In case if you are too lazy to put your time and strength in making something great by doing a small difference, just recall about these set of people who dedicated their precious time and effort, doing researches on CGI, just to comfort your eyes even for a 1/30 of a second. One pixel on your computer screen, alone, may not contain a significant value. However, millions of those together, can make characters which a whole world would probably fall in love with, unconsciously.
Exposition Magazine Issue 15
Department of Industrial Management
University of Kelaniya