Faces, faces

Technology is changing the way we see (and recognize) people

Look at the above picture montage. Notice anything unusual? What is completely unique about each of the portraits? They are very nice, but why am I showing them to you?

I’ll tell you: none of the above persons actually exist. The portraits were generated by a machine learning system that has been trained on a large number of (genuine) photographs and can now generate new pictures that look perfectly authentic to human observers. Or can you spot the artificiality of the above images? Click to enlarge, and look carefully. Are you able to see any discrepancies to reality? I bet you cannot.

If you want to see many more similar portraits, in equally stunning and realistic quality, pay a visit to the ThisPersonDoesNotExist web site. At the bottom right, displayed when each image is shown, are links that take you to the next image, but also give you code and take you to places that describe the process involved in generating the images. You can also view a large number of hi-res artificial faces here. Use your mouse wheel to scroll through them.

Here’s another example: I have inserted a picture of a real person. Can you tell which that is?

Of course the above is state of the art (2020) AI image generating technology, but if you want to start experimenting in the field you can use MakeHuman, a free and open source 3D computer graphics software that will allow you to generate photorealistic humanoids. It is written in Python and is freely available in GitHub.

This is what the MakeHuman interface looks like. It is fun to use.

Face recognition

Like me — and most other folk — you have probably given up bulky cameras and are using the pea-sized lens of your mobile phone to shoot better images than with the Canon or Lumix that now reside unused on the shelf. The photos are automatically uploaded and stored in Google Photos, where your family and friends can see them, seconds after they were taken, even though they may be on the other side of the globe.

The stored photos have full information on when they were taken, and where. Typing “Hamburg,” “Munich” or “Chennai” will display all pictures from those locations. But there is another function I recently discovered:

Google will show me a face and ask me to identify the person. Then it will show me the same person in random images from my photo database, and ask me to confirm if they are of the same person. In the above example it is even showing me a tiny image reflection of myself in the glass window taking the picture (in the middle). The face recognition software is truly awesome. After confirming ten or so images it knows me, and searching for “Frederic Friedel” will show all pictures in which I appear.

Recently a company called Clearview has started using face recognition on a truly grand scale. They have compiled a database of over three billion faces and identities, scraped together from Facebook, YouTube, and thousands of other social websites. The explicit intention is to help law enforcement track down criminals, pedophiles, terrorists and sex traffickers. It is also used to help exonerate the innocent and identify the victims of crimes like child sex abuse and financial fraud.

Until now, this kind of technology has been regarded with great suspicion because of its potential to radically erode privacy. Some large cities, like San Francisco, have barred police from using facial recognition technology, but hundreds of law enforcement agencies have already started employing Clearview “to identify child molesters, murderers, suspected terrorists, and other dangerous people quickly, accurately, and reliably to keep families and communities safe.”

It is interesting to know that the Clearview sofware could easily be paired with augmented-reality glasses. Just imagine: you would be able to identify every person you saw, shoppers in a mall, people at a party — but also attractive strangers on the subway. You could immediately see their names and where they live, what they do and which their interests are. Brave New World, most of us justifiably fear, but what if you could also instantly spot that the man handing out sweets to a child is a known pedophile, or the one following you has a criminal record for robbery and rape?

Deep Fake

Another recent and very alarming development is Deepfake —a name given to the use of machine learning to fake videos. The AI learns what a face looks like at different angles, and overlays it on an actor as if it were a mask. Hollywood has used this technology to bring dead actors back to life.

You can see exactly how Peter Cushing was generated in this report.

But deepfake has more sinister uses, and we need to be aware of what is coming. Take a look at the following video:

This is not a genuine video of Barack Obama speaking. It’s AI that knows how his face and head are shaped, and how his lips move when he is speaking. And it generates a sequence like this that never actually happened.

Below is a fairly comprehensive explanation of Deepfake technology. There are many tutorial on YouTube, but I will not link to them — we don’t want to help people to take up deepfake as a hobby (or for revenge porn).

The first third of this video explains how deepfakes are made, and the last third (from 15 minutes onwards) how the Planet of the Apes creatures are filmed, and how the giant visual effects of the movie were created.

The final section of the last video has tempted me to add the following section:

CGI—computer generate imagery

This is just an addendum — I will write about the subject in a future article. Computer generated imagery uses AI to create scenes or special effects in films. And this has completely changed the movie industry.

Take the epic 1963 film Cleopatra, directed by Joseph L. Mankiewicz and starring Elizabeth Taylor, Richard Burton and Rex Harrison. Above is a scene from Cleopatra, which received nine Academy Award nominations, including for Best Picture, Best Art Direction, Best Cinematography, Best Visual Effects and Best Costume Design.

All the sets you see in Cleopatra are real — made of wood, cardboard and plaster. And all the thousands of people, all the soldiers, citizens and town folk are real human beings, background actors hired for the scenes and dressed up as Romans or Egyptians. That explains the production cost of $44 million (half a billion in today’s dollars). Initially the film lost money at the box office.

What are epic film productions like today? Take a look at a truly massive scene from Lord of the Rings, with many thousands of warriors:

Or look at this scene from the Chinese-American production “The Great Wall.”

Not a single human being or structure in the above image is real — everything is computer generated. There are imposing edifices, thousands of soldiers armed with spears, using bungee cords on the wall to impale monsters from Chinese legends. This cannot be done with real actors or real sets.

Here are some of the best CGI scenes from the past ten years — and here some of the worst (but quite hilarious). Finally take a look at what is possible today in computer generated imagery (click to enlarge)

Not a single pillar or roof slate, not a single tree or leaf, not a single ripple on the water actually exists. They are all computer generated!

All this can easily mean the end of set building and even of human actors?! I fear that might be the case. On the subject of CGI and movie animation you might also like to read my article Godzilla — Men in Suits.