Jason Toy

What Is The Goal of Building AIs?

2021-07-10T00:00:00-07:00

What Is The Goal of Building AIs?

Published Sat, Jul 10, 2021

You’d think with the possibility of the Singularity wiping out humans, we’d be more careful and try not to advance so fast.

There is clearly a strong emphasis of AI in pop culture, government, scientific research, and many other facets of life. Where does this fascination with AI come from? I want try to explain our obsession with wanting to create an Artificial Intelligence.

There are several camps of people trying to bring about artificial intelligence.

Computer Science

Computer Science (CS) is the study of computation and algorithmic processes. One way I like to think about CS is about turning processes and everyday phenomenons into computationally efficient representations. The first Computer Science Degree was creted in 1953 at the University of Cambridge and since then CS has become one of the most widely studied topics in the world. There are a very diverse set of subfields such as operating systems, data structures, algorithms, cryptography, databases, computational chemistry, computational neuroscience, computational physics, compiler design, and of course artifical intelligence. No matter what area a computer scientist focuses on, artificial intelligence will always come up. To a computer scientist, AI is the pinacle of computer design and engineering. With building an AI, you could potentially build anything instead of having to hand craft it yourself. Many computer scientists dream of building intelligent computer systems that can solve any computational problem. Often the problems a computer scientist studies in university are very different from real world software engineering problems.

Philosophers and scientific knowledge seekers

People think of AI as a field since basically the early 1900’s, but in many ways the field is actually a lot older.

In our oldest recorded history there have been philosophers questioning the very nature of ourselves and intelligence. Why is it that we can communicate with others, yet animals cannot do so at the level we can. Humans can use multitudes of tools, think about the past and future, recognize themselves, while most other animals can’t do any of these things. Over the thousands of years of recorded human history, we have been contemplating and theorizing about how we come to be.

For example, René Descartes published some writing to show that animals were mechanistic robots that had no emotions which shaped how society treated animals. Many philosophers and scientist want to fundamentally understand how our mind works and if it is fundamentally possible to create intelligence inside of a computer. As long as the human race still exists, humans will continue to try and understand how we are different and how our intelligence works. We did not have computers until the 1900’s, but we have been trying to understand these questions using philosophy. Creating AIs helps us to better understand ourselves and all living beings. AIs will ultimately let us know if we are truly unique or just some freak accident in the evolutionary chain that can be replicated and improved.

Life extensions and Immortality

Most people dream of living forever and for some of the rich who have made their money through technology ventures, many of them are trying. There are billions of dollars being invested into companies and technologies that could make us immortal such as cryogenics which can freeze our brains and bodies for later revival, stem cell preservation, brain scanning technology, limb replacement technology, brain computer interfaces (BCI), and extreme forms of diets are some of the technologies being developed. AI is seen as a key component to helping us achieve immortality in multiple ways. AI technologies help researches find answers faster to current scientific problems. For example, in 2021 Google DeepMind released AlphaFold2, an computational model that can predict protein structure. Proteins are large complex molecules that are essential to all life. Almost all functions that our bodies perform rely on protein and so if we can understand proteins, we gain a deeper understanding of life, biology, aging, and disease. By knowing the shape of a protein,it’s role within the cell can be estimated and then drugs can be developed that work with those proteins. Typically the way scientists have examined the shape of a single protein is to use advanced technologies such as X-ray crystallography, nuclear magnetic resonance and cryo-electron microscopy. Those experimental technologies are all error prone and cost a lot of resoures, both time and money wise. To see the shape of a single protein structure can take up to years of works and cost millions of dollars. At Seung Lab and Eyewire, we worked on building neural networks that scan brain images to reconstruct a human brain network, called a connectome. The connectome is a complete description of an organism’s structural connectivity of the brain and nervouse system similar to how the genome maps all the genetic material of an organism. Think of the connectome as a wiring diagram of the brain. Due to the massive size and complexity of the human brain, it is not currently possible to scan and store the whole connectome for a single human brain. It would require more storage than currently all the hard drives on earth combined to just store the data. So instead, most research groups focus on cutting out specific brain tissue regions the size of a few microns.

Capitalism, businesses, labor, and algorithmic efficiency

Business are constantly trying to improve efficiency and lower costs to generate more revenue. If one company finds a technique to lower costs they can then sell to their customers for a cheaper price and generate more revenue. Other companies will try to copy those techniques creating a never ending battle for who can generate the most revenue. Computers and technology have accelerated this trend even more. Companies that are using technology to compete try to develop more sophisticated algorithms and this has led to using more AI and machine learning as a competitive advantage. If you are a company competing with other companies, you are always looking to be more efficient and replacing human labor is looked at more often as technology improves. One of the earliest times this happened is when textile factories replaced humans who hand weaved textiles with machines that could weave faster and cheaper. Amazon, the online store that sells everything, uses AI and robots to move packages between warehouses, which replaced humans from having to manually move packages across the warehouse. Notably, as of 2021 Amazon or any other company has not figured out a reliable way to use robots to get packages into an individual box destined for a customer’s home. That is due to not having enough technology and understanding of the sensorimotor system to create robotic hands that work like human hands. Almost every large company that has phone support has replaced all or most of their human support specialists with AI computer systems that can talk and understand basic concerns a customer might have. This reduces the cost of having to employ people to man the telephone system. Microsoft, the operating system creator, has implemented an AI technology called CoPilot which gives code snippet suggestions as a software developer types into their code editor. Think of it as super version of spelling and grammar check, but for code. This will allow for faster and cheaper software products to be produced, which will reduce the amount of software engineers employed.

With capitalism, one of the most expensive costs for a business is human labor: salaries are expensive, they can be unreliable, finicky, get sick , need to be managed, and have other problems, while a computer can keep going non-stop and won’t complain.
Capitalists want to take as many human employees out of the job , so they have fewer workers to pay and manage. As long as capitalism’s model of “cheapest cost no matter what” is the main paradigm, this is the path that technology and capitalism with create.

Racing to save the planet and the human race

Many people such as Stephan Hawking and Elon Musk, believe the human race is in a downward spiral. We are overpopulating the earth, consuming all resources, destroying all the habitats and ecosystems of all living things, which will eventually lead to the destruction of the earth and the human race. The thought is that artificial intelligence could potentially save us. Our scientific knowledge and acquisition moves slow compared to the rate at which humans are utilizing and destroying the earth’s resources. If we could develop an AI, it could potentially learn much faster than us and help us solve all kinds of problems more efficiently such as:

how can we feed the growing human population
how can we generate enough energy to power all of humanity’s needs in a clean and environmentally safe way

Government and protecting countries

Governments around the world are constantly trying to gain a competitive advantage against each other economically and/or with military power. During the 2010’s, The United States and China have been developing AI’s as a continuation of an algorithmic arms race. Think about the race in the 1960s to the moon between the US and Russia. They were fighting each other to see who would be the first to make it to the moon and that fight caused them to push each other and accelerate their technology. Even with a public race, governments are constantly researching new technologies and AI is an are that the US government has spent a lot of time on in for example autonomous vehicles, life long learning models ,symbolic reasoning, real time machine learning, and more. Its not just about competing with other countries,China is widely known to collect massive amounts of data relating to its people to train models that are to do full facial scanning of its whole population and give social credit scores to every citizen. AI is the next level of technology advancement for governments to grow and gain a competitive edge as many technologies such as deep learning show lots of promise. Vladimir Putin, president of Russia at one time, predicted that whichever country leads the way in AI research will come to dominate global affairs. postal workers

As long as the human race exists and true AI hasn’t been created, there will always be people working to solve AI. Humans’ competitive nature and curiosity into the mind along with the never ending technological progress will cause humans to never stop working towards this goal. Only if some large catastrophe will humans stop working toward this.

* money/ power
* solve science
* Eliminate workers/ jobs
* Immortality
* Computer science goals
* getting off the planet
* futurists

What Exactly is The Singularity

2021-07-07T00:00:00-07:00

What Exactly is The Singularity

The technological Singularity (often called the singularity for short) is a term that signifies a potential future where artificial intelligence surpasses human intelligence. After that point in time, civilization would radically change into a future that no human could comprehend. The idea of the singularity was first written about around 1950 by John von Neumann, one of the founders of computers, phycisist, and a polymath. He was paraphrased saying: “One conversation centered on the ever-accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue.”

Later in 1993, a computer scientist and science fiction author, Vernor Vinge popularized the idea of the singularity with a paper titled “The Coming Technological Singularity: How to Survive in the Post-Human Era”. In the paper he discusses aritifial intelligence with exponetial growth,technological unemployment, human extinction, intelligence applification for humans, and the unpreventability of the singularity. He really brought to center the idea of a doomsday scenario following the singularity where computers don’t need or want humans around and this idea has now entered the mainstream consciousness. In his paper he writes: “What are the consequences of this event? When greater-than-human intelligence drives progress, that progress will be much more rapid. In fact, there seems no reason why progress itself would not involve the creation of still more intelligent entities – on a still-shorter time scale. The best analogy that I see is with the evolutionary past: Animals can adapt to problems and make inventions, but often no faster than natural selection can do its work – the world acts as its own simulator in the case of natural selection. We humans have the ability to internalize the world and conduct “what if’s” in our heads; we can solve many problems thousands of times faster than natural selection. Now, by creating the means to execute those simulations at much higher speeds, we are entering a regime as radically different from our human past as we humans are from the lower animals.” Vernor believes that this shift would drammatically change the time scales of our future: “From the human point of view this change will be a throwing away of all the previous rules, perhaps in the blink of an eye, an exponential runaway beyond any hope of control. Developments that before were thought might only happen in “a million years” (if ever) will likely happen in the next century.” Vernor argues that if the singularity is even possible, the singularity cannot be avoided. That is because the lure of power it offers; competitive advantages in economics, military, and technology is so compelling that even if governments where to forbid these developments, it would just means someone else will get to it first. Further in his paper he talks about an alterantive path to the singularity called Intelligence Amplification (IA). This is scenario is a lot more safer for humans. Instead of independent computers evolving, humans are modified and amplified with computers to allow us to improve our memory retention, information processing, and other biological processes. These computer interfaces would act as subfunctions of human processes instead of running fully independently. Instead of trying to replicate human intelligence, researchers could focus on building computer systems that support human biological functions that we have already been studying and have some basic knowledge of. This would require much further advancement in our understanding of human biology, especially the brain. We would also need to accelerate advancements in human computer interfaces (HCI) to allow these technologies to properly interact with our bodies.

In 2005, Ray Kurzweil, a computer researcher, inventor, and technologist,wrote a book “The Singularity is Near”. In the book Ray tries to paint a picture of evidence showing that technology and artificial intelligence is growing at an exponetial rate and that our computer technology is just at the start of that growth curve. The result of all this growth will create something unrecognizable to anything we have ever seen. Ray talks about the possibillity of AI’s that merge with humans and nanotechnology that could transform us into countlss untold ways. He walks through charts and examples showing this growth trajectory which cumlimanates in his prediction of the Singuarity occuring around 2045. He believes that this technological Singularity will allow humans to achieve immortality, whether digitally(consciousness uploads) or physically (biological life extensions). Many of the arguments Ray make sense, but if you dig deeper into some of it, there are glaring holes.

With all these ideas of doomsday AI scenarios and potential immortality, the idea of the singularity has enterted into the mainstream zeitgeist. Countless movies and tv shows have been created showing what could potentially happen at the point when AIs become super intelligent. The Terminator series, West World, Avengers: Age of Ultron, Transcendence, 2001: A Space Odyssey, Her,Blade Runner,Ghost In The Shell, The Matrix, and so many more. Almost all of those movies end up with humans attacked, killed, and at the bottom of food chain. Her is the main exception from the list, the AIs become super intelligent and find that the humans are not smart enough and boring and so they all leave to another place (perhaps another dimesion) where humans can’t reach.

Some scientists, such as Stuart Russell and Stephen Hawking, have expressed concerns that if an advanced AI someday gains the ability to re-design itself at an ever-increasing rate, an unstoppable “intelligence explosion” could lead to human extinction. An AI’s brain would be a computer and not biological, and so would not have the limits of humans biological brains to grow and replicate. Humans are currently the smartest species known to exist and look how we treat other species below us: we eat them, hunt them for sport, use them as tools, or just plain try to kill them if they annoy us. If a computer would become smarter than us and could make its own decisions, there is no telling on what it would do. It could theoritically kill us because we are annoying or they could use as energy like batteries.

On the other side of the spectrum, some people thing we would end up in a much better situation where instead the computers become partners with us, merge with us, or just continue to be our subordinates and do what we tell them to. Computers would be intelligent, but for some reason may not have the same propencity for war and destruction as humans, and so may be fine working with humans. Image having computer doctors that could see our diagnosis and instantly recognize any ailment and have a customized cure for us. Imagine having a computer parner that you could ask any question to any problem and have it instantly solved in the most efficient way possible. Our world would have millions of Einstiens, geniuses that could invent new technologies for us nonstop. In this future, many people invision us accelerating our own biology to cure disease and aging, advancing our technology, and bringing a better future closer to us.

We have no way to know what that future would look like, but its not stopping people from trying to redirect us from hypothetical doomsday scenarios. Scientists,researchers, and futurist are trying to set humanity on a correction course by building ethical standards for the research and creation of AIs. The Machine Intelligence Research Institute (MIRI) who does AI research does AI research and states: “Given how disruptive domain-general AI could be, we think it is prudent to begin a conversation about this now, and to investigate whether there are limited areas in which we can predict and shape this technology’s societal impact….There are fewer obvious constraints on the harm a system with poorly specified goals might do. In particular, an autonomous system that learns about human goals, but is not correctly designed to align its own goals to its best model of human goals, could cause catastrophic harm in the absence of adequate checks.” Ethical AI is another group trying to help redirect the development of AI: “The current and rapid rise of AI presents numerous challenges and opportunities for humankind. Gartner predicts that 85% of all AI projects in the next two years will have erroneous outcomes. Adopting an ethical approach to the development and use of AI works to ensure organisations, leaders, and developers are aware of the potential dangers of AI and by integrating ethical principles into the design, development and deployment of AI, seek to avoid any potential harm.” The Future of Life Institute states: “Artificial intelligence (AI) has long been associated with science fiction, but it’s a field that’s made significant strides in recent years. As with biotechnology, there is great opportunity to improve lives with AI, but if the technology is not developed safely, there is also the chance that someone could accidentally or intentionally unleash an AI system that ultimately causes the elimination of humanity.” The Institute of Electrical and Electronics Engineers (IEEE) has a subgroup called The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems that has the mission of “Ensuring every stakeholder involved in the design and development of autonomous and intelligent systems is educated, trained, and empowered to prioritize ethical considerations so that these technologies are advanced for the benefit of humanity.”

The Algorithmic Justice League is trying to create equitable and accountable AI: “The Algorithmic Justice League’s mission is to raise awareness about the impacts of AI, equip advocates with empirical research, build the voice and choice of the most impacted communities, and galvanize researchers, policy makers, and industry practitioners to mitigate AI harms and biases. We’re building a movement to shift the AI ecosystem towards equitable and accountable AI.”

AI4ALL is a nonprofit working to increase diversity and inclusion in artificial intelligence: “We believe AI provides a powerful set of tools that everyone should have access to in our fast-changing world. Diversity of voices and lived experiences will unlock AI’s potential to benefit humanity. When people of all identities and backgrounds work together to build AI systems, the results better reflect society at large. Diverse perspectives yield more innovative, human-centered, and ethical products and systems.”

OpenAI seems to have the biggest ambitions and most funding (~2 billion as of 2021): “OpenAI is an AI research and deployment company. Our mission is to ensure that artificial general intelligence benefits all of humanity. OpenAI’s mission is to ensure that AGI-by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.”

Along with all these institutions being created, universities are also creating groups with similar goals of saving humanity. The University of Oxford has created the Leverhulme Centre for the Future of Intelligence (CFI). Their goal: “Our aim is to bring together the best of human intelligence so that we can make the most of machine intelligence. The rise of powerful AI will be either the best or worst thing ever to happen to humanity. We do not yet know which. The research done by this centry will be crucial to the future of our civilisation and of our species.” The University of Berkeley has created the Center for Human-Compatible AI (CHAI). Their stated mission: “The long-term outcome of AI research seems likely to include machines that are more capable than humans across a wide range of objectives and environments. This raises a problem of control: given that the solutions developed by such systems are intrinsically unpredictable by humans, it may occur that some such solutions result in negative and perhaps irreversible outcomes for humans. CHAI’s goal is to ensure that this eventuality cannot arise, by refocusing AI away from the capability to achieve arbitrary objectives and towards the ability to generate provably beneficial behavior. Because the meaning of beneficial depends on properties of humans, this task inevitably includes elements from the social sciences in addition to AI.”

There are dozens of other groups working on redirecting humanity and AI from these potential doomsday scenarios. Clearly there are a lot of smart people concerned that this is a big problem right now and worth trying to figure out before its too late.

In the following writings, we will discuss where AI technology currently is and whether this is really needed or not.

On Grounded Language Learning and Simulation

2020-12-22T00:00:00-08:00

On Grounded Language Learning and Simulation

Published Tue, Dec 22, 2020

Language seems to be the pinnacle of intelligence. While many animals can communicate, none can do it at the richness or level of complexity as humans. So its not surprising that we have studied language in so much detail trying to understand how language works and what makes human unique in their use of language. There has been a lot of research trying to build computers that can understand and process language in ways similar to a human. Some of the main fields are Natural Language Processing (NLP), computational neuroscience, computational linguistics, artificial intelligence and machine learning. The recent fruits of these labor have resulted in many software architectures such as word2vec, fasttext, CBOW, transformers etc.

These software architectures are so powerful and have become state of the art for text classification, voice recognition, and text generation. And with these new possibilities, many people seem to believe that true computer intelligence is eminent. GPT3 just came out in June 2020 and because it can generate text that looks like it was written by a human, many people think that computers can actually understand and communicate with us now. What has always concerned me with these computer models is that they have zero understanding of the actual world and are just glorified puppets, or as they are called in philosphy, philosphical zombies. And so what ends up happening is that we continue to build models of intelligence on a broken foundation. Yes, these new models are so powerful now that they can hear the words we are saying and can accurately transcribe human speech at 95% accuracy, generate human looking articles, but these computer algorithms still have no idea of what any of it means.

All the initial concepts we learn stem from interacting with our environment, they have physical meaning. A baby’s first concepts come from interacting with the world via touching, falling, grasping, crawling, feeling pain, etc. Although they can’t initially say these words, they do have some understanding of using these concepts to interact with the world and change their sensory inputs via their actions. Their internal representations, the meaning of these concepts, therefore must have some representation that captures the interaction between themselves and their environment, they must contain some form of embodiment, or as they call it in the cogntive science field, embodied cognition.

Just think about what these common words mean to you: up, balance, behind, ahead, heavy, noisy, smooth, and rough. Those words all have their meaning derived from our interactions with the environment and as our understanding of the world grows, the way we use concepts grows into more abstract forms. For example, the concept “balance” is first understood as standing upright in the world and distributing your weight and gait in a way that you don’t fall over. As your knowledge grows, you learn that you can apply balance to any kind of physical object and then later apply it to any abstract thing like balancing multiple responsibilities.

Another example, the concept “rough”. We first learn about the concept “rough” from touching certain physical objects like wood, sandpaper, cement, etc. “Rough” means to have uneven or irregular surface. Later, people learn to apply “rough” to things that are not completely finished like a rough draft or apply it to situations that didn’t finish effortlessly like a “rough night” or a “rough trip”. The concept “rough” is used where more physical exertion must be applied than normal or expected.

Contrast that with the concept of “smooth”, another concept learned from our interaction with the physical world. “Smooth” means having an even and regular surface. We first learn about this concept from touching objects like walls, ice, certain plastics, etc. Over time we apply it to movement such as walking and then move up the abstraction ladder and apply it to any situation that was done without effort, ie there was less physical exertion applied than normal. I had a smooth date or the interview went smoothly.

Another example, the concept “crumble” is to break into small fragments. As kids we play with toy blocks and sand and we see that we can collect them, stack them, and watch them fall apart due to gravity . Over time we learn to apply this concept of falling apart to more and more abstract concepts: the building crumbled, our relationship crumbled, NATO crumbled, etc.

Lets move on to abstract concepts. The definition of “proposal” is to put forward an idea for consideration. So you take something that others don’t see or know about and you move it, typically with your hands, in front of everyone so that they can view it. So there is a physical component involved whether you are proposing something in person or over email. The concept “to impose” is to force something to be put in place. Impose is always used in an abstract sense like imposing taxes or imposing your opinion on others, but to understand the concept “to impose” you must understand concepts like “to be forced” and “to be blocked”, which are all centered understood from the physics and integrations of our environment.

We can keep going over the root of these concepts, but the main point I want to get across is that EVERY concept we understand has some portion of its meaning derived from interactions between ourselves and the physical world. Meaning is grounded in our physical world from the need to survive. These interactions with our world are important to learn cause and effect. With every interaction with our world and daily objects, we learn the results of us exerting our will onto objects. If I push a rock over a ledge, it will fall and may kill my prey. If I move my legs fast and continuously, I will move faster, my heart rate will increase, and I may escape from this lion. If I knock over this cup of liquid, the floor gets wet and slippery. If I move this switch, it becomes lighter or darker in the room. Without all these interactions, its NOT possible to understand more abstract concepts like homework, deadlines, proposals, and calculus.
As of to date, our machine learning models have no way to store these interactions, nor do we have the data available in a meaningful way for a computer to understand it. Just parsing large corpuses of text is not enough to understand. As adult humans, we are often thinking in abstract terms, but without grounded knowledge in the real world, it makes no sense and that goes the same for computers.

Lets look at how computers store representations. Word2vec, a famous deep learning language reprentation model, in more detail. Each word is represented as a vector where each element in the vector is a number from 0 to 1 representing how close another word is to the main word. So for example, the vector for the word “king” will have words like “queen”, “royalty”, and “crown”, near it and so they a number close to 1, while words like “dog”, “haircut”, and “window” will most likely get assigned a number closer to 0. The algorithm is self surpervised, it parses large bodies of text and is able to automatically build its model and so needs minimal help from a human to learn. From only reading text, this model learns and stores no information about the physical world, no interactions, only what words are typically near it.

In the image below, you can see what the vectors look like for a dataset of 5 words.

So, If you try to find any sort of meaning in those vectors, you will essentially be going in circles, as each vector just leads to another vector round in round like a merry-go-round.

Given all this, I’ve been contemplating on how you could train computers to learn grounded meaning, how could we teach computers about our physical world,what would the data structure look like, and what would you need to build this. To be honest, I don’t really know, there are many possible directions and there is a growing body (no pun intended) of research in the space.

When we talk about interactions with our world, we are talking about perceiving the world through our senses. Most animals, including humans have access to 5 broad categories of senses: touch, taste, smell, vision, and hearing. When we think of touch, we usually think of fingers, but our whole skin is a touch sensor and is also the largest organ of our whole body. Our sense of touch is actually compromised of many types of sensors. Our bodies have proprioception receptors, which sense where all of our body parts are in relation to each other, temperature sensing neurons, pressure sensing cells, merkel cells which detect different textures, head direction cells and more. Neuroscience is still very early in understanding all the types of sensors and cells our bodies have.

So if want to teach a computer how to interact with our world, it would make sense that it would need some kind of sensors. Computer algorithms have a form of vision and can process images and videos very efficiently. But for the interactions we have been talking about, vision does not seem to be the right modality to capture the richness of perceptual experience necessary to ground meaning. In fact, contrary to trend of most machine learning research, I don’t believe vision is necessary at all. Helen Keller has led me to believe that the massive blob of sensors we call touch is the key area of focus. Due to an illsness, she was blind and deaf starting from 19 months of age and lived until the age of 87 years. She had no vision and hearing for practically her whole life and never had those sensory experiences in her conscious life! She wrote several books and her favorite of mine is “The World I Live In”. If you didn’t know she was missing her senses, you would not be able to tell. She discusses what consciousness is like and what words mean to her. For example, when she thinks about the concept “duty”, she imagines a straight line, rigid and never leaving its course. “When I think of hills, I think of the upward strength I tread upon. When water is the object of my thought, I feel the cool shock of the plunge and the quick yielding of the waves that crisp and curl and ripple about my body”. She contributes her intelligence to her hands and sense of touch. I highly recommend you read her book.

So with a computer, how would you give it access to all these sensors? You could try building hardware: robots that interact with our world via sensors. If you go down this route, you will run into many problems. It is slow and expensive to do a “build and test” cycle as you need to physically build your robot. The sensors you give to the robot may be inaccurate, wrong, or most likely don’t even currently exist to measure the things we may need. And so every iteration of the robot you build may be missing critical functionality and to develop the right version is just not practical, so that leaves software.

You could build software simulations of a world and your character (typically called a learning agent in machine learning). Video games do this every day with games like Grand Theft Auto, World of Warcraft, Doom, etc. These games all have simplified physics engines that mimic parts of our reality, ignore other parts, and exaggerate others. With any simulated software, you need to pick and choose which of the parts of the world you need to simulate accurately because your computational power is limited. Do you focus on gravity, collision detection, friction interactions, accurate light interactions, biochemistry reactions, temperature, etc. The list of possible physical phenomenons to model in simulation is too long and we have no idea what is required for intelligence to emerge. On top of that, building these physics engines is a very labor intensive process, not some tiny project. So we have to go with an open source engine like Bullet or PhysX.

Anyway, so I wanted to experiment with building a dataset that could teach physical interactions to a computer. Instead of thinking about it for too long and getting stuck, I wanted to start experimenting with a live system immediately. So….. I took a children’s dictionary, collected all 4000 words/concepts, put them into a (spreadsheet)[https://concepts.jtoy.net/concepts], manually analyzed all the words, and sorted them in different ways: part of speech, physical interaction level, abstraction level, etc. I then took 100 words that I thought made a good representation of core language concepts along with some harder concepts and coded up mini simulations that displayed their “definition” of the word as little cartoons. With this dataset I have collected, I consider each word from the children’s dictionary to be interchangeable with concepts. I put all my work online in a project I call “ConceptNet”. I focused mostly on physical action words (verbs) versus nouns, adjectives, and abstract verbs. Each simulation uses basic shapes such as circles and squares to represent bodies. Each simulation is programmed with some randomness, so they are different every time you view them. I created 6 distinct environment types to represent the concepts: a 2D overhead world with physics, a 2D world without physics, a 3D world with physics, a 3D world without physics, a 2D sideview world with physics, and a 2D sideview world without physics. Many of the simulations are crude and basic, but hopefully clear.

I wrote all the code in Javascript so that the results could be viewed by anyone with a browser. I used a library called p5js to do all the drawing, matter.js for the 2D physics engine, and XXX for the 3D physics engine.

Here is the simulation for the verb “to leak”:

Here is the simulation for the verb: “to meet”

Here is the simulation for the verb: “to rotate”

In building this proof of concept (POC) I was able to explore a lot of ideas, I learned a lot, and now I have even more questions. I’ve written about somof my findings and next steps below.

Parts of Speech

In English and many other languages, the main parts of speech (POS) are nouns, verbs, and adjectives. While reading through the children’s dictionary, I noticed many words have definitions in different parts of speech. For example, “complete” is an adjective and a verb. Black can be used as a noun, adjective, and verb. The color black, describing the color of something, and to make something black. Most words I looked at had definitions for both verbs and nouns. For example “repeat” is used as a verb:”to do something again” and as a noun: “an event or action where something is done again”. Another example is the concept “cover”, as a verb it means “to put something on top or in front to protect or conceal it” and as a noun it means “a thing which lies on or over something to conceal or protect it”. Many of the words and concepts we learn transfer easily from action to object and vice versa. There are rules and abstractions that allow us to “stretch” the definition out so we can use these concepts in different ways. A computational data structure would most likely store definitions that would allow fluidity and stretching. In writing simulations for adjectives, I found that adjectives are just really a way to show things that are different from normal, where oftentimes you use yourself as your baseline for normal. So when we say someone is fat, we are saying they are fatter than average or fatter than myself. The same for smart, brave, calm, eager, and every other adjective. So to draw these in simulation, I showed a transition between 2 states: what is considered normal and second state showing the different adjective. For verbs I always showed an agent moving around and performing the verb. For nouns, I tried to show the object in a situation where its specific properties where highlighted. For example, for the concept “cliff”, I showed other objects on a cliff that would fall off the cliff.

Abstractions

The definition of “to complete” is “to make something whole or perfect” or “to finish making or doing”. The simulation I did for “to complete” is very specific, it shows an agent collecting boxes to form a much larger box. I think it is as clear as possible given the constraints, but with all these simulations, we know they are crude simplifications. The holy grail for these models it to be able to store an abstract representation. Computer models are too rigid and do not have the flexibility that humans seem to possess. When I look at the simulation for “to complete”, I wonder how could a computer represent this in an abstract way. Is it simply a combination of its sub definitions: “to make” “something” “whole or perfect”? Is it stored as a clear distinct entity or as a web of meaning? Completing something can require a variable amount of time, ie completing your PhD takes longer than completing your homework,which takes longer than finsihng your food. It seems like the concept “to complete” can be applied to any kind of situation like homework, food, puzzles, or anything with a goal or ending. Its as if the mind learns a general computation for “to complete” that can run on almost any other input it receives similar to how a computer circuit is built to do specific computations for certain data types.

Points of views

Since I made many of these simulations with basic primitives, I often had many of the simulations look the same for many words. For example “to guide” vs “to follow”. Both of those words involve multiple parties where one is guiding and one is following, so it depends on which one you are focusing on to describe who is doing what. If you focus on the guide, then they are guiding, and if you focus on the follower, then they are following.

When describing a situation, every person will usually describe it a little differently because they focus on different things. For example if you see 2 people are fighting, you may choose to describe the situation with the words: to fight, to attack, to punch, to hurt, to beat, etc. Those words are all similar and can describe the situation.

choosing which aspects of the world to model

I spoke about the difficulty in modeling physics earlier. I was unable to simulate a few concepts because the physics engine wasn’t powerful enough. “To fold” would require a physics engine that modeled bending on hard bodies (like metal) or soft bodies(like skin or cloth). I also could not showcase any concepts relating to liquids or air like “to dry”, “wet”, “to irrigate”, “to inflate”, “to puff”, “to blow”, “to breath”, “to boil”, “to paint”, “to rinse”, and related concepts because the physics engines I experimented with didn’t support these.

Our computers are not powerful enough to model everything in our world. So instead, we carefully pick and choose certain aspects to focus on in and implement. This is not just true for our model, but for all computational models. For this prototype, I just wanted to get a sense for what would need to be developed further. With the concepts I focused on, one area that I kept seeing was related to friction. The concepts “smooth”, “rough”, “pressure”,”to dodge”, “getting crushed”, “to bend”, “collision”, “to ascend”, and “to fall” are all have concepts related to friction. Friction is a physical phenomenon, it is a measure of the resistance from one object encountering another object. In all those examples I gave and in everyday life, there is often an implied resistance. When we think of resistance, we often think of heaviness, difficulty, pain, slow, or heat. So Our theoritical data structure would need to account for some sort of friction measure.

Emotion

Many concepts are words related to emotion such as “to ache”, “to apologize”, “to argue”, “to astonish”, “to appreciate”, “to cherish”, “to love” and more. These type of emotion related concepts are argubly some of the most important words to humans. Look at all the cherished works from Shakespear as an example. Even before we had working machines we have been debating on whether it is possible for a computer to have emotions. So there is no way we will come up with a fully working model within the scope of this project. I did find an interesting point though. Going back to friction, it is often thought of as a negative concept. The more friction that is involved between objects, the more potential for fire, parts of the objects tearing off, moving slow, and heat being generated. If your own body is involved, then this friction will mostly likely feel like pain. The concepts of liking or disliking something could be modeled after the physical phenomenon of magnetism where objects have different magnetic fields which cause certain objects to attract and repel eachother.
The concept “to decline” means to decrease or get worse. Decrease has a physical basis of going down or becoming less, but why does it have a negative basis of “getting worse”? Is it because the opposite of decline, “to increase” is usually associated with positive situations like growing or having more? The main point I want to make is that we theoritcally could implement some crude form of emotional meaning by associating emotional concepts with physics based counterparts the same way we are doing it with all the other concepts. Check out this emotion model built from neuroscientists studying the common housefly.

Where to go from here?

So I built out a POC and learned a lot. I didn’t build the data structure and algorthm yet, but that is the main goal and so experimenting with that would be the next step. I want to continue writing more simulations and turn this into a living dictionary. Longer term I imagine this dataset could be used as a computer parseable wikipedia like Cyc. Or a standalone computer book to teach computers/humans like from the SciFi book The Diamond Age. I purposely skipped doing any machine learning with this, but that would definitely be future direction to explore.
Check out all the work I did here and if any of this is interesting to you, please contact me.

Deep Reinforcement Learning is a waste of time

2019-12-30T00:00:00-08:00

Deep Reinforcement Learning is a waste of time

Published Mon, Dec 30, 2019

Its the end of 2019 and I wanted to see how AI progress has been doing. Deep Reinforcement Learning (Deep RL) in particular has been hyped as the next evolutionary step towards Artificial General Intelligence (AGI), computer algorithms that can learn to do anything like humans in a general way.
Researchers have been working on Deep Reinforcement Learning (Deep RL) for a few years now with incremental progress. The idea and hope around Deep RL is that you could easily train an agent to do theoretically anything like drive a car, fold laundry, play video games, clean a house, solve a rubix cube, etc all without hand holding the learning process. Some of these experiments have resulted in some success, you can teach a Deep RL agent to play some video games and board games, but transferring anything to the real world, into production systems, has mostly been a failure. As of now, the end of 2019, I still do not know of any success stories for production system that use Deep RL.

Deep RL systems have multiple problems, I am not going to go into the details of them, there are a bunch of blog posts that already talk about these problems

I wouldn’t be surprised if at least some of those problems improve or get resolved, like finding more abstract ways to represent information inside of neural networks, but it doesn’t matter if you can’t solve what I consider the core issue: manual reward engineering. In Deep RL, I would say most of time is spent on engineering your reward function to get your agent to do what you want. The reward function is the objective function in more traditional machine learning terms, the algorithm uses this to know if it is heading in the right direction, the more reward it gets, the “better” the model does.

For example teaching a robotic arm to fold laundry, lets say you have a pile of pants, how do you write a reward function to fold them properly. It sounds easy when explaining this to another human, its just “fold the paints into neat piles” but a computer has no idea what any of these rules mean. For each experiment, you have to program your reward in a way that the computer can measure progress on its own, all without having any understanding of what its actually doing. So you might start off giving rewards for touching the pants, then more points for grasping the pants, or maybe points for moving the pants. How do you reward for the actual folding part? Does it get points for having 3 folds? How about points for no wrinkles? You end up spending so much time trying to guide the agent to follow the right path that its basically turned into a fully supervised problem. You theoretically could have a human watching the training process and then the human could assign scores for each action the system takes, but this does not scale. These Deep RL systems need to run anywhere from ten’s of thousands to millions of iterations trying out every variation of actions to figure out which are the right sequences to achieve their goal, a human could not possibly monitor all the steps the computer has taken. There is active research into trying to make this scale with Imitation Learning, but to me it’s just trying to improve something that is fundamentally broken.

During your journey of trying to train your laundry folding robot, you may need to constantly adjust the reward function as it accidentally rips the pants, drops the parts, folds them while being inside out, or folds them in weird ways that don’t seem to make any sense. Reward engineering turns into an experimental process of trial and error to find what works. There are countless documented reports of Deep RL models doing all kind of unexpected movements.

Reinforcement learning is typically classified as a separate 3rd class from supervised and unsupervised machine learning, but in my mind its currently really supervised learning. The current process is: you start training the model, then you watch it fall over and die, then you keep tweaking the reward function over and over for a very long time until maybe, just maybe you get a desired output. What part of that is not giving it training data? You’ve just convoluted the whole process and made everything more difficult by having to feed the answer in a non direct way to the agent. If computers are going to learn from their environment, it must happen in a 100% unsupervised manor.

So why has there been so much hype around Deep Reinforcement Learning?

If you look at Deep RL from an abstract level, it is depicted as an agent that over time, learns from its environment. That seems absolutely correct and does mimic reality, all living organisms starting from birth learn how to live and navigate from and in their environment. We know this because we have done many experiments that empirically show us that if we change and constrain the environments for newborn organisms, they learn different things and behave differently. Deep RL is very different from traditional machine learning methods like supervised classification where a program gets fed raw data, answers, and builds a static model to be used in production. The part that is wrong in the traditional Deep RL framework is the source of the signal. These artificial agents must learn directly from themselves and their environments, not some artificial reward function that we feed to them. When DeepMind came out with a single Deep RL agent that learned how to play Atari video games with almost no intervention, it was thought that this could scale up to all kinds of problems in other domains and potentially even AGI. What we’ve learned is that most things organisms and humans do, CANNOT be modeled as video games with agents optimizing for gaining as much reward as possible. And don’t be fooled by Deep RL winning games with no manual work, the reward engineering still happened, it just happened 40 years ago when the video games makers hired whole teams of people to make these games. Lots of people made this mistake thinking Deep RL was the next step in AGI, even Google was overzealous and spent $500+ million to buy DeepMind in the hopes of taking AI to the next level. It seems like the community latched on to the idea of Deep RL being the holy grail because it is the closest form of machine learning to sort of mimic our own reality, even though in reality its not even close. What we’ve ended up with is lots of parlor tricks where little AI toys can play all kinds of video games, board games, and not much else.
The main benefit of going from more traditional machine learning methods to Deep Learning was that manual laborious engineering has been removed. You can theoretically give the model a bunch of data, run your optimization algorithm, and it would learn without you manually writing feature extraction code. So Deep Learning has been applied to part of reinforcement learning, mostly on the agent side taking in perception data in the form of pixels, but if you are still spending most of your time hand engineering the system, then in my mind its still doesn’t work. I didn’t even go into detail of the other Deep RL problems like if you need to slight modify the objective, then get ready to retrain your whole system or that slight variations in the environment usually ends up in complete failure… AGI is on the way…….

So now we have the top machine learning research institutes, DeepMind and OpenAI, still spending the majority of their time and resources on Deep RL. They are constantly bombarding the internet with PR pieces showing the great progress they are making: Deep RL can play Go, StarCraft, Dota 2, solve a rubix cube, etc.

I am confused as to why they continue to spend so much time on building Deep RL systems that beat games with clearly defined rules and point systems. I think everyone gets it now, with enough time, money, and computers almost brute forcing every single possible action, Deep RL can beat almost any game. I believe a large portion of their strategy is to just pump out PR pieces with smoke and mirrors showing how the latest Deep RL system can beat the next best game so that they can continue to receive investments from people who don’t know better, just look at the recent $1 billion Microsoft gave OpenAI or the continued access to the Google piggybank for DeepMind. One of DeepMind’s recent successes (~5 years after the Google acquisition) is that they trained a single Deep RL agent to play Go, Chess, Shogi and Atari games. Yay, more video games!!!!!! I’m not trying to beat up on them, I’m really happy they are around and actually making an effort to solve AGI. The problem is that they are creating LOTS of false perceptions which ends up wasting a lot of human brain power on dead ends and leads the general public to think we are lot closer to AGI when everyone in the industry knows its nowhere near. Secondly and more importantly, they are sending the wrong signal to the AI research population to spend more time on Deep RL. Many researchers and hackers look up to these companies and see these PR floods coming out of from them and spend countless hours hacking away at those same problems when they could be focusing their energy on bigger and more fundamental problems. If AI and AGI are going to move forward, I think its time to stop playing video games and focusing on harder problems.
Fortunately the DeepMind and OpenAI do spend time in other problem areas, but as I said, it seems like Deep RL is still their main focus.

Current Deep RL systems seem like they could be a good fit for systems that already have basic training. Think transfer learning where the agent already has a basic understanding of its environment and of itself. Watch a human child from birth, at first its starting to learn to see shapes and colors, move its fingers, touch objects, control its body, differentiate sounds, learn about gravity, physics, falling, squishiness, bounciness, learn object permeance, etc. All of that is happening to various degrees in every single living human and organism. Usually after a human baby has acquired a very large repertoire of knowledge: it can walk, grasp objects, use the bathroom, do basic talking, etc. Then more formal training happens like sending a child to school where it then goes through a more structured process of learning: homework, grading, and quizzing to train the child to learn from the curriculum. Current Deep RL systems seems like it could work really well for this kind of formalized training where the desired goal can be clearly and mostly automatically graded AND after the agent already has a basic grasp of its environment. If the child can’t recite ABC, fail, if they can recite the whole alphabet, pass. Currently Deep RL is putting the cart before the horse though, we are trying to train computers from scratch to do complicated tasks that may sometimes work, but because these models are trained for specific tasks versus generalization, they have no generalized understanding of their environment and end up with systems that are too brittle and don’t work very well. No amount of newer learning algorithms like back propagation, DQN, PPO, DDPG,TRPO are going to fix Deep RL as long as the source of the reward function is not fixed. I’m sure we will make incremental improvements and maybe get a few more points out of Starcraft, but its not going to make big efforts towards the holy grail of generalized artificial intelligence without a fundamental shift in architecture that allows the agent to learn in a fully unsupervised manner from its environment.

My own hypothesis is that the reward function for learning organisms is really driven from maintaining homeostasis and minimizing surprise. Homeostasis is the process that living things do to maintain stable conditions needed for survival. Every living organism must survive and maintain its body separate from the external environment. When its hungry, it feeds, when tired it sleeps, when thirsty it drinks, when hurt, it rests, when hunted, it evades, when too hot, it tries to cool down, etc. These innate and primordial signals tell the organism what to do and what to focus on as it navigates the environment. Failing to do these things results in failing to maintain homeostasis and ultimately an early death. As the organism is navigating around, it builds a model of its own sensorimotor interactions together with the environment as it interacts with its world. Its model starts to put things together: when dark, it should be cold, when I burp, I should hear a sound, when I move left, I expect to see vision to shift by X, as an animal walks in front of me, I expect it to continue moving forward, as I brush my finger on a cup, I expect to feel a smooth surface, when I make sounds through my vocal chords, I expect to hear a corresponding sound in my ears, etc. If anything unexpected happens, something that deviates from its model, then the model readjusts its predictions until it gets what it expects, which may result in more moving. If that fails, then genuine surprise occurs and those neurons are marked for “requiring model updates”, in which relearning and re-optimization may happen either in real time or while the organism is sleeping. Optimization occurs between the agent’s internal model of the environment and the actual environment and runs continuously to minimize surprise. These ideas have been discussed in neuroscience and philosophy for a long time by people like Andy Clark with his ideas on predictive processing and Karl Friston and his free energy principle. To my knowledge, I have not seen these ideas successfully deployed in machine learning environments yet. I believe there are many technical issues: how to mimic homeostasis in a computer, how the internal representation of the model should be stored, the low resolution between agents’ sensory apparatus and the environment, and low fidelity environments, among other issues.

I did not mean to write this article with a flamebait title, but I couldn’t figure out a way to explain to people that the current Deep RL architecture is fundamentally flawed. We don’t need more models that can play more video games, we need to implement models that can learn generalized representations between the agent and their environment that can be learned in a fully unsupervised manor. This new architecture may be called Deep RL 3.0, active inference, predictive processing, or something totally different, but please stop wasting time on this dead end. Deep RL research has come up with some cool and interesting work, but its time to move on. Whoever can figure out a way to build the next general of AIs that can learn from their environment in an unsupervised way is going to make a HUGE difference and move the field forward.

offline manifesto - offline in an online first world

2018-04-20T00:00:00-07:00

offline manifesto - offline in an online first world

I manage projects with lots of people involved, write software, constantly communicating , contemplate, and generally need to be online quiet frequently. Over the years, I’ve noticed I’ve had productivity swings based on several factors, one of the main ones being whether I was online or not. With the constant email notifications, slack messages, and other incoming messages, I found that often times I got very little completed. In fact, I have found that being online constantly has been detrimental to my well being. Not just from notifications, but I would find my self going to visit news website subconsciously in the middle of my work. The more often I would be online, the more I would see the quality of sleep go down and I would find myself waking up in the middle of the night. I think many of you have these same feelings. The book Deepwork explains in detail why things like the internet take away from your ability to do deep work and how to battle this, so please read that book if you want to understand more. One interesting quote from the book:

Spend enough time in a state of frenetic shallowness and you permanently reduce your capacity to perform deep work. “What the Net seems to be doing is chipping away my capacity for concentration and contemplation,” admitted journalist Nicholas Carr.

So for the past year, I have been experimenting with ways to be able to get into deeper states of concentration, given this internet connected world we live in. After experimenting with many different configurations, I think I have found a middle ground that works for me and allows me to get into a state of concentration to produce deep work.
Basically I spend half of my day offline (includes cell phone and computer), with “breaks” where I batch all the work I need to do online. During my “breaks”, I do all those online tasks like research, emails, download software, sync data, etc.

I’ve had to change most of the software I use to enable this process and it is working out fairly well now.

Computer and Software

My main computer is my laptop and is has everything I need to be productive. I try to leave my computer disconnected from the internet as often as possible. When I am working in an office with other coworkers, I will leave my computer online, but I will use an app called Focus, which blocks all non wok related software. In today’s world, there is so much information and information is key to a lot of people who do knowledge work. I keep a copy of wikipedia on my laptop. If there are any concepts I need to understand, I can search through wikipedia. Any notes or todo items I have, I will put them on simplenote which is a simple note taking app that syncs across all my devices. For anything I need to do when I get back online, I write it in my offline todo list and then when I am back online I will do those items.

For my mail, when I am online I process all of my email through gmail, but when in offline mode, I use mac’s mail.app and I keep a copy of it synced on my computer through IMAP. That way I can still read and write emails.

I keep my books synced through amazon kindle on my laptop and my kindle device so if there is information I read and want to keep, I can get it through my device.
Often times I need to look up documentation for the different programming languages and frameworks I use. I have a program called Dash which stores the documentation for various programming languages locally. I can easily look up and search the standard documentation.

I have to read a lot of scientific research for my artificial intelligence research project, so I keep dozens of papers on my computer and I store them in an app called Mendeley. Its basically just a a GUI for searching, collecting, and marking up PDFs.

I keep hundreds of different software repos on my laptop that pertain to any type of project I am working on. I use this as reference code.

mobile device

My phone is setup to be even more extreme. I keep it in offline mode most of the day. Even when it is online, I have all notifications turned off. I don’t have email on my phone. My phone acts as my notes app via Simplenote, camera, music player, maps, and uber. I know its extreme, before, I would check my phone constantly and now I rarely have the urge to check my phone and I’m more present in whatever I’m doing, so I think its worth the tradeoff.

Offline Manifesto

So I have this process working mostly ok, but what I have found is that a lot of software I was using is designed to work only online. Some of this software is created purposely to only work online so as to keep you on their website as long as possible. This creates a reinforcing cycle of anxiety because you are required to always be online. I think software should instead be setup so that everything works in offline mode by default. That means you can use the software whether you are on a plane, in a rural area with no internet, or offline and the software will still function. Then when you get back online, it all syncs up to wherever it needs to go.

Even with all the ubiquitous internet, we need to be able to work offline. As we go deeper into the oceans and further out in space, an always on internet connection seems unlikely. I bet that if we make it onto other planets hundreds of years into the future , software will all work in an “offline mode first” system allowing us to be fully productive when offline.

Why you should learn graphics programming

2017-05-03T00:00:00-07:00

Why you should learn graphics programming

I’ve been programming in a lot of different areas for over 15 years now: websites, system admin/devops, database applications, backend worker processes, machine learning pipelines, marketing automation tools, and more. I would dabble with graphical programming environments like Processing and shaders occasionally to draw with a computer, but never really got into it. It wasn’t until recently that I’ve been doing machine learning with image processing that I got really into graphical programming.

Graphical coding has a few different names. Some people call it creative coding, graphics programming, generative art, or poetic computation. They all mean slightly different things, but the common theme is that they all use computer code to generate graphics. Creative coding usually means coding for expressive purposes versus functional purposes. My friend Taeyoon Choi doesn’t like to call it creative coding because it implies that other types of programming is not creative and I agree with him. For this post, I will use the terms creative coding and graphical programming interchangeably, but it applies to all computer generated graphics. Regarding creative coding languages, I mostly use p5js and Processing, but there are many other graphical programming environments like GLSL and WebGL. So anyway, after I got into creative coding, I had this aha moment where I thought creative coding should be taught to everyone and that it will help make you a much smarter person. So I’ve sort of become an evangelist for creative coding and I talk to a lot of new programmers these days about it. I often hear “creative coding seems cool, but how is it useful for real work?” I think all programming is great to learn, but I want to talk about why creative coding in particular is important and I’m going to try and answer that in 2 parts, why learning creative coding is useful, and what kind of work do creative coders do.

Part 1 - Why creative coding is useful

Math and STEM Math is a key subject to learn throughout your schooling, no one can argue with this. I studied a LOT of math growing up, so much that I ended up studying it for my college degree. What is ironic about that is I didn’t get a chance to use it much as a programmer as I was spending a lot of my time just glueing things together. When I got into creative coding, I saw it as a great way to really test out my intuition with math. I found it like being a kid in a candy store, I went through many of my old math books and took functions and algorithms and drew them to the screen and played with them. I still study math in my spare time and I can honestly say that it has accelerated my understanding of many math concepts from linear algebra, to calculus, to physics and much more. Basically all graphics programming is literally writing math. That might sound scary to a lot of people, but before you form too many opinions, I implore you to try it out, take an area of math that you want to have a better understanding of, try to draw some aspect of it with code, and then tweak some of the variables related to your drawing. Just by being able to tweak the variables and seeing the output, you will build stronger mental models of how the math works. This doesn’t apply to just math, but anything from STEM- science,technology,engineering, and math. And If you didn’t learn from your interactive drawing, at the worst case, you just created your own artwork :)

Simulations and Models

Graphics programming is often used for simple simulations and building models. What exactly is a simulation? According to wikipedia it is “the imitation of the operation of a real-world process or system over time”. Simulations are often visual, like a city simulation, forest simulation, or ocean simulation. In more scientific areas, a simulation is called a model. You could build some simple simulations for research purposes or just for your own understanding. I’ve built some very basic simulations for studying intelligence. If you are studying natural phenomena or natural sciences, you might be able to apply creative coding to your research. These simple simulation don’t even have to be super accurate, sometimes its good enough to just see a simple simulation of some problem you are thinking about and that can lead you into insights or other ways to look at your problem. An early version of an ant simulation I built Teaching Using creative coding as a teaching tool is very powerful because so much of our learning and understanding is visual. Writing programs to be run from the command line can be scary to many novices. Having instant feedback from visual coding enables a user to quickly visualize connect the input of code to the output. Plus the user can then tweak the variables and see what each piece of code does. Most importantly, creative coding allows for a lot more FUN, because you can really play around and create new and beautiful things. Sometimes programming can get pretty boring or tough. If you can increase the fun factor, it will keep your student(s) much more engaged and likely to continue push through their struggles. If you have to teach new programmers, I would highly recommend you start off on creative coding before moving to more traditional methods of programming. I have a dream to one day create a full computer science curriculum using creative coding libraries.

Becoming a better programmer

I think its safe to say that computer science is an important skill and more and more people continue to learn how to communicate more directly with computers. There are so many different areas in programming that most people cannot learn all of them. If you can spend more time programming with more mind bending ideas and different paradigms, it will make you not just a better programmer, but a better thinker. I have spent time trying to learn the more weirder and different programming concepts like Lisp,erlang, neural networks, smalltalk, monads, and it has really helped expand my thinking. I think that creative coding is definitely one of those concepts you should learn to expand your mind. Whether you are just getting started in programming or you have been programming for years, it will help you immensely if you can take some time to try creative coding.

Part 2 - what kind of work do creative coders do

Teaching

I mentioned teaching earlier, you can go out and teach creative coding as a job. There are many universities that have a creative coding curriculum, its usually part of the media arts or computer science programs. Most large universities have this like NYU, MIT, Carnegie Mellon University, and more. A lot of high schools also teach an intro to computer science class. If you like teaching, this could be a cool job.

Advertisements and Creative Agencies

I have a love and hate relationship with advertising. Some people consider it a necessary evil, advertising can be considered to be at a unique place, it is at the intersection of technology, money, and mainstream consciousness. Usually its creative agencies who use creative coding to create new advertisements. So people who are working here can be involved in some very cool campaigns. There are a bunch of cool creative agencies like R/GA whose focus is combining technology with storytelling. Many of these agencies deploy creative coding technologies to create interactive installations using different media formats like TVs, projectors, LEDs,etc.

Storytelling

This has some overlapping with advertising, just without the ads :) Storytelling is done across many mediums like comic books, tv shows, and movies. I have not personally seen a lot of creative coding being applied here, but I know its being done. I thing there is a lot of fun work that can be done here and I’m surprised creative coding is not being used more here. If there are people using creative coding here, I would love to here from you. I’m working on a side project with some friends that is a mini web series using all code to make the animations.

Video games

Video Games are basically just another extension of simulations. You probably already know about video games, so I’m not going to go into any detail here. Its a natural jump to go from graphics programming to video games. Games often do help people relax and in some cases improve human performance such as hand eye coordination. I’ve heard it can be a grueling job with long hours, but it could be fun.

Any programming job

So I see creative coding as a gateway drug to doing general programming and once you are able to program in one domain or language, it gets much easier to switch to other programming languages and domains. For example: after learning to draw with a computer, then it might make sense to work with external web services so you can make your sketches get data from the web. Or going from Processing to graphics shaders is a logical jump. So if you keep programming and trying new things, eventually you build understanding around it and you could get any programming job with enough time and practice.

Data visualization jobs

Data visualization is increasingly important as we get more and more data to process. The job of the data visualizer is to take data and make sense of it, and present it in a way that the audience can understand. There are many companies who are dealing with vast amounts of data and need different ways to summarize and show their data. Typically a data visualization job would go to analysts, graphics designers or data scientists. This is a tough and rewarding job and requires strong analytical skills.

Art

Being a full time artist is the dream of many people. It is a very tough road and no guaranteed success. Still, many people attempt this route and I commend them for it. I know of a few creative coders who use languages like Processing to produce artworks to sell such as Tyler Hobbs Using creative coding here could mean creating completely digital artworks or even physical objects that are connects with computers like art installations. There are large thriving communities of digital artists in places like New York City and San Francisco.

Other jobs?

I would love to hear from you if you are a creative coder for your day job and doing something different from what I’ve listed above.

Closing thoughts

I hope I’ve convinced you that at least dipping your toes into creative coding can be mind expanding, fun, and potentially help your career. I personally think it is the easiest way to get into programming. I want to see more young people getting into programming and creative coding. Since I started creative coding, I have been drawing new sketches every few days. I put most of them online here: http://jtoy.net/sketches I also created a challenge site that sends out a weekly challenge for you to try and draw: http://pushpopchallenge.com

My experience with Processing

2017-01-04T00:00:00-08:00

My experience with Processing

I have seen and dabbled with Processing a couple times over the past few years, but it wasn’t until I was in a computational art program that I spent real time with it. Over the past few months I tried to write a new “sketch”, a program in Processing terms, every day to play with a different aspect of graphical programming like shaders, color modes, 2D motion, arduino integration, kinect interactivity, image pixel manipulation, and more.

I have to say overall I love what Processing has done and so I’ve collected my thoughts and experiences of using it over the past few months. I’ll go over technical issues, user experience, and my hopes for Processing. To help understand my point of view, I will give a little history of myself. I’ve essentially programmed a little bit of everything, dozens of web apps, machine learning pipelines, sales and marketing tools,devops and system admin, backend systems,deep learning models, and now graphical programming. I also study programming languages: clojure, common lisp, ruby, python, bash,javascript, php, C, erlang, haskell, java, prolog, smalltalk; the weirder the language, the more interest in it I have.

Searchability

In terms of the name Processing, it is a terrible name for searching online. Processing is such a generic term and is used for every type of computation and so it is extremely rare when I search for Processing with any other phrase, that I get what I want in the first try. Many other languages like ruby or python have this same problem,but I would say nowhere as severe as Processing. With p5js, things get a lot easier search wise. Tip, if you can’t find info in google, switch out Processing for p5js.

freezing editor

The editor completely freezes on me several times a week. If I type something wrong,access variables that don’t exist, cause the memory to shoot up, or other mistakes, often times Processing will completely die or freeze and I have to force quit it. On top of that, sometimes I have forgotten to save my code and I end up loosing some of my work. This sucks, I wish the editor was not tied to the sketch running. This ends up with me wasting a lot of time. I usually end up coding with 2 terminal windows, one for vi and the other for running my sketch and it runs flawlessly and faster.

gallery mode?

So I wrote lots of sketches to display at a showcase. I wanted to have each sketch run for a few minutes, then go on to display the next one, essentially running in “gallery mode”. So I googled for it, coudln’t find anything, it doesn’t exist. So then I went to ask a few Processing experts, they all said, “yes, this is common, its very easy to do, all you do is modify all your code to fit into one master sketch and just write a little code to switch between sketches” Are you kidding me? I have to change all my code around to present my sketches? I ended up writing a simple command couple line script so you can place all your Processing sketches into a directory and the script will rotate between them every X seconds. Gotta love the power of unix! Since each sketch is reloaded with a fresh java process, there can be a noticeable delay of 2-3 seconds between sketches depending on the speed of your computer. You might want to use a background image on your desktop to convey to the user to wait a monement.

poetic computation

I think most coders try to write things as succinct and efficiently as possible. With Processing, I saw myself taking this to another level and that is because when writing Processing code, you are essentially writing applied mathematics. As a web programmer or backend programmer, you just don’t spend as much time using math. And so as I wrote my Processing code, I found myself trying to rewrite it to be more elegant and more mathematically pleasing. I find myself trying to remove as many if statements as possible. Often times this leads to more efficient code. Processing has rekindled my love for mathematics.

computer science and math education

In school I studied mathematics and computer science and since then I have always been interested in figuring out how to get more people interested in learning to program. I’m a firm believer that programming can solve everything :) I think Processing could be a wonderful tool for teaching computer science. Processing definitely has problems as a teaching language, but I think its better than most of the stuff out there. Imagine teaching students to use fundamental computer science and math algorithms by writing them in Processing and then for homework having students tweak various aspects of the algorithm and show the results visually.

I don’t know if it exists, but I would love to see someone come up with a full computer science curriculum using Processing as the main language.

on java

Processing is written on top of java. I would say that is both a good thing and a bad thing. I personally loath java, its so painful for me to write in that last time I had to program in java, I immediately learned clojure so I could avoid java. For Processing, I understand why they went with java, Java was a good choice because they could build on top of an existing system, its cross platform, and there are tons of libraries to integrate with. The bad side is that the code is verbose,ugly, and slow to startup. People don’t enjoy writing in java. I was tempted to just start writing Processing code in jruby or clojure, but didn’t want to add another level of abstraction and so far have stuck with java. I wonder what Processing would look like if it was written from the ground up as a new programming language. Would it be as successful as it is today? I think there is still room for visual programming language written from scratch.

Processing vs p5.js

So p5.js is Processing written on the javascript runtime. That means you can easily write and deploy your Processing sketches to the browser. When I started this adventure a few months ago, I was trying to decide whether I should use Processing or p5.js. In hindsight, I’m pretty sure I made the right decision, p5.js is not as powerful, you can’t integrate it with as many libraries, it is much harder to debug your code, it is not as mature as Processing, and it seems to run slower. Even with the small sketch I wrote here of the abstract buddha, I originally wrote it in Processing, works fine, then I ported it to p5js. In safari it runs fine, but in chrome it slows down the whole page and looks like it freezes for a minute at a time. Power of computation was the main factor in my choice, so if you need raw power, Processing is the better choice. So how does one choose the right version to use? Are they both going to remain actively developed? Can they do the same things? Is p5.js the successor? It is a little confusing, other people have asked this same question in the Processing forums. And from the Processing foundation’s point of view, I hope they can focus on one of these great platforms, not divert limited resources to both.

gateway drug

So over the past few months, I did some tests where I walked through writing a Processing sketch to 3 types of people: programmers, techies who don’t code, and non techies. I wanted to see if I could get them excited to do more programming. I saw the most excitement from programmers and some excitement from techies. Not exactly the level of enthusiasm I was looking for, but I think my pitch and content need more fine tuning. Part of the feedback I got was, “this is cool, but what can I use this for?” To me, I think what the question really means is “how can computational artists make a living” which is a topic I’ll write about later. I think with the right content, Processing could get a lot more people interested in visual coding and coding in general. The content needs to progress from very simple to progressively challenging, all while remaining visually pleasing. As I mentioned in the beginning of this post, I had already dabbled with Processing before, but at those times I just wasn’t really interested. So what was it that got me excited about it this time? I think it was becauseI had an upcoming computational art show and I had been doing image processing with deep neural networks. I think things like raspberry pi and Processing combined could help get people interested in this space.

raspberry pi

So Processing is supported on the raspberry pi. It is still very early stage though, it runs extremely slow on the raspberry pi and you have to follow some special instructions to get it working properly, making it a bad experience. For an art installation I was working on, I tried to use raspberry pi, Processing, and a webcam. The code was too slow and webcams don’t seem to be supported, so I ended up using a mac mini. In my tests, 95% of all the programs I ran are too slow. It is not uncommon to have code that ran at 60 frames per second on your mac to run at 3 frames per second, unusable! I think Processing and Pi could be a powerful force to get people interesting in visual programming, but the way it is now is a missed opportunity. I found lots of optimizations and tricks to make your code run faster and I plan to write up those notes in a separate blog post. I keep a raspberry pi with Processing loaded in my backpack now to show people! I would absolutely love to see a future where raspberry pi can be used to teach and deploy Processing artworks. Keep in mind you have to write in Processing, p5.js doesn’t really work on the raspberry pi :(

daily sketching and closing remarks

I have tried to write a new Processing sketch once a day, but usually ends up at ~3 sketches a week. I find writing a sketch to be like an artist practicing painting. It is fun, therapeutic, and helps you improve your coding practice. I want to help get more people involved in visual coding, so My friend,Agustín Ramos Anzorena and myself are working on a site called pushpopchallenge to practice writing a sketch everyday. Several times a week, there are different challenges where you can practice visual coding. One day it might be try to do a ocean simulation and the next day it might be to draw meteoroids. I hope it can help you and give you inspiration to paint with code.

If any of this sounds interesting to you, please contact me to collaborate.

the machine learning challenge at apple

2015-09-23T00:00:00-07:00

the machine learning challenge at apple

if you want a machine learning challenge, go work at apple

Apple has recently joined the machine learning and deep learning craze. They are hiring at an accelerated pace and competing to hire form the same talent pool as all the other Silicon Valley companies.

They have a unique set of challenges compared to companies like google. Apple has championed privacy as one of their core beliefs. That means building out the same features as many other companies, but without sharing data back to Apple or other 3rd party services. Services like recommendations, search, and voice assistant need to work from the device as opposed to over the cloud. That affords the end user access to these helpful services, while not comprising their privacy by sending all the data to the cloud.

This probably presents a challenge for Apple when hiring as software developers want access to as much data as possible when building their models. That is part of the allure of working at Google, unlimited access to all their data. I wonder how often their copy of the internet is processed to test out new models. I would say that working at apple is an ambitious and worthy challenge. For their machine learning models to work, they need to get their models to run fast and efficiently directly on their devices. That means these algorithms have access to less resources and less data compared to a company like Google. In the long term, it could mean the creation of newer and better algorithms that can run on their constrained devices. Working at Apple on these machine learning problems could mean breakthroughs as they are coming at these same problems as everyone else, but from a completely different angle.

What I find innovate about Apple is they are masters of creating efficient hardware to solve current real world needs. Their A series chips that power all their iOS devices are always the best in class ARM chips. Every time they release new iOS devices, their chips outperform all the other devices. I would wager that apple will develop dedicated chips to power their deep learning initiates if they haven’t done so already. To do efficient deep learning, you currently need fast concurrent floating point calculations which means graphic cards. There is one company currently who provides graphic cards that are great for deep learning applications and that is NVIDIA. And even with NVIDIA chips, efficient still means weeks of training to build their machine learning models. Right now NVIDIA basically has a monopoly on deep learning hardware. Working at Apple could foster more competition and innovation in deep learning hardware.

neuroscience based startups

2015-09-04T00:00:00-07:00

neuroscience based startups

04 September 2015

I wanted to see what kind of neuroscience based startups are being worked on these days. I looked at all the companies on angel list ( a fundraising platforms for startups) that were tagged as neuroscience. Angel list has about ~600,000 companies and 106 of them are tagged under neuroscience. I looked at all the neuroscience tagged companies. Out of that list of 106, there were 12 that seemed interesting. It looks like to me angel list is not a good place for finding neuroscience based startups. A lot of neuroscience companies in the list are dead, I’d say about 30%.

Here are the companies that seemed interesting: https://angel.co/senselabs - hardware to improve brain functioning, interested to see if it works https://angel.co/branching-minds - applied cognitive science to learning software https://angel.co/mint-labs - brain imaging in the cloud, are people using this? https://angel.co/personal-neuro - looks like they pivoted, but looks interesting https://angel.co/re-compose - I would do this software on an iwatch https://angel.co/backyard-brains - teach kids about neuroscience with hardware! https://angel.co/halo-neuroscience-1 - hardware! https://angel.co/synetiq - marketing tools https://angel.co/neuroclick-1 marketing tools https://angel.co/cognitive-algorithms-laboratory - mad scientist? https://angel.co/efficio-s-r-o - cloud storage for EEG data, they should build a database around this data https://angel.co/neuroinitiative - simulations? https://angel.co/about-my-brain-institute - performance at work https://angel.co/neurontools - hardware and marketing

Here is a thread on quora that lists some other interesting neuroscience based startups: https://www.quora.com/What-are-some-interesting-psychology-neuroscience-startups

I know there are a lot more neuroscience based companies out there. The Boston area has a lot of brain and science based companies because of the concentration of colleges: http://eyewire.org https://naralogics.com/ https://www.neumitra.com In my research I came across a neuroscience based accelerator: http://neurolaunch.com/

I was expecting to see a lot more neuroscience based companies. A lot of the companies that are under the neuroscience category on angel list are not really neuroscience companies. They use machine learning techniques to extract some data or use concepts from cognitive science to train you to be smarter. This was more of an exploratory search for me. I want to understand how the intersection of neuroscience and profit looks like. I was looking for companies that had the potential to open up neuroscience to more people and accelerate our learnings of the brain.

Helen Keller and somatic intelligence

2015-08-29T00:00:00-07:00

Helen Keller and somatic intelligence

29 August 2015

who is helen keller

Hellen Keller was a woman born in 1880. At the age of 18 months she lost her hearing and vision rendering her blind and deaf for the rest of her life. Despite that, she was still able to be an intelligent and productive member of society. She interacted through the world with touch, taste, and smell. I’ve had a deep interest in her ever since I learned about her history. The story of her learning her first “word” or named concept is amazing. At around the age of 7, a teacher was brought in to teach her how to communicate. The teacher would try to give her objects to touch and then write the words in her hand. This went on for a while as Helen didn’t understand that every object had a word to identify it. The breakthrough came when the teacher was running cold water on her hand, and then writing water on her hand repeatedly. After Helen realized it was a word, she demanded the name of every object she could find! From there she went to school and became an active member of society. She wrote many books ,lectured around the world, and was an ambassador for people with disabilities. I’ve read some of her work and it was amazing how she could use such poetic language to describe her thoughts and her world all without ever hearing or seeing anything.

can we still learn without all of our senses?

In normal human society, vision and audio sensory input is the primary way we teach and learn. As a baby, our parents are constantly talking to us and that is how we learn our first words. We see our parents moving their mouths and we copy their movements to speak our first words. We learn the alphabet by being shown the letters in books and written on paper. At the same time our parents are speaking the words to us so that we associate those sounds with the words and concepts we learn. As we grow older we continue to learn from watching videos, learning from our peers, and reads books. This continues to use audio and visual sensory as the medium to communicate this information. Now image you have never heard or seen anything, how does that feel? The way we would communicate with the world would be by touching and smelling things. Most of your time would be by yourself “hearing” your inner voice. What does an inner voice sound like if you have never heard anything? Lets say you recalled in your mind a physical object like a chair.

You would recall how the letters are spelled on your hand. You would recall how a chair feels when you sit on it. You would recall what a chair’s purpose is and how it relates to other concepts. You would know that there are many types of chairs that come in all different shapes and sizes.

How would you learn what the concept of “now” means, this very moment. It is a concept that you learned from interacting with your environment and eventually learning about the concept of time. You would probably think of lots of other concepts like sleep, weeks, days, breakfast, lunch, resting, energy, time,etc.

What would recalling the concept of “friend” feel like? A friend is someone you spend time with and you find an interest in that person. You continue to interact with that person and spend time with them. That person might make you feel good or happy and so you enjoy spending time with them. If you thought about the unique features of that person you would remember the texture of that person would your hand touched them. You would feel the word friend on your hand.

These are all concepts and memories that she had, very similar to us, but embedded in her mind and neurons in a completely different way than we think. Or is it actually the same?

focus on vision in machine learning and intelligence algorithms

How we teach computers to learn is more similar to how most humans learn than how someone like Helen Keller learns. We feed in mostly representations of images and raw text into a computer. Depending on the kind of machine learning problem the computer is trying to solve, it optimizes for one of several types of output: trying to differentiate between a positive and negative classification predict a numerical value based off the input group the input with other input that you have already seen. A common way to see how smart our machine learning algorithms are is to see how well it comprehend handwriting from the MNIST dataset. DeepMind, an artificial intelligence company building models to watch video games to learn how to play was bought by google for $600+ million . We are training computers to distinguish between males and females by studying the different features of their faces and bodies. We already know the mind contains concept neurons . They light up in a brain when the concept of something is brought to its attention. For example the concept of star wars, you have a group of neurons that lights up overtime you hear the theme music of star wars, you see yoda, you see the name star wards written, you see parody movies, anything related to the concept of star wars.

Neural networks, especially deep neural networks are hot in the machine learning community now. They supposedly learn from general data, but are best known for processing general data.

common algorithm for learning

Learning about how Helen Keller becoming a member of society makes me believe that the the way the brain learns is through a common learning algorithm that learns specifically how to interact with its environment. The body can add any new sensors to the body and as long as the brain can process that data through neurons then it can use and learn from that sensory data . That could be radar, the ability to see infrared, the ability to sense certain chemicals, etc.

We think Audio, Vision, Smell, and taste are very different, but they are just sensory inputs into the brain. Touch is considered a separate sensory system from the others. In science and medicine nomenclature, touch is referred to as the somatosensory system. It is a whole system because touch is made up of several inputs: pressure, temperature, skin,vibration, and probably some others we don’t fully understand. You could even go as far to say that the other common sensory inputs we spoke about are just other ways to interact with the environment as they are just subsets of the somatosensory system. Almost every single living creature in our world has the ability to interact with the environment and remember some part of its interactions. That is what learning is. I believe that for our computers to actually learn, they must learn by interacting with its environment and storing that information in a way that relates memories based off past experiences. I would like to see agents learning from sensory data that do not have vision.

self comes to mind

A lot of these ideas came to me while reading Hellen Kellers works and from a Antonio Damasio, a cognitive neuroscientist who wrote a book called Self Comes to Mind . Antonio Damasio believes the brain maps body states to interact with its environment a that emotions are preprogrammed autonomous programs that get executed in the body to help us survive our world. In his books he talks about a high level cognitive architect that the brain runs. I have never come across computation models that mimic the somatic sensory system. I believe that any system that wants to accurately model the brain must model a general sensory system which means the agent must have a body. Reverse engineering the body’s algorithm could result in a general artificial intelligence that learns similar to the way we do.

conclusion

Building a somatic sensory computation model could potentially “solve intelligence”. Building computation models that build based off version are the wrong direction. Yes, the brain and eyes are doing visual pattern recognition to see lines and edges, so it is a form of learning, yet that is not the core of intelligence. Learn how the inner brain is processing all of the sensory data regardless of where it comes from and we can unlock the key to intelligence.

I would love to contribute to this and see it built out in an open source fashion.