Questions to be considered:
Main Concepts:
Additional:
The increase in capabilities of AI systems in the past few years has led to both excitement and fears. Models that can generate convincing text, visual and audio are now possible on consumer grade machines, showcasing the surprizing effectiveness of our current learning algorithms. It seems the estimate by optimistic experts of a 50% chance of AGI before 2030 is now quite plausible depending on your definition of AGI. This was also predicted by the father of computer science and artificial intelligence Alan Turing:
It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers… They would be able to converse with each other to sharpen their wits. At some stage therefore, we should have to expect the machines to take control. ― Alan Turing
The author likewise agrees that as there are no strong evidence that the human brain is special beyond the physical realm, it is quite plausible that human-like AGI may arrive in the next few decades. Current generative models may be an important part of future AGI, but seems to lack the vital element of independent curiosity which the author considers a requirement for Independent AGI.
This essay focuses more on the positive perspective of how Friendly AGIs have a shared interest with humans, in contrast with the more prevalent dread some may, rightfully, feel with some impending AI systems. The author makes a distinction between (1) AI systems (including non-independent AGI), and (2) Independent AGIs, suggesting that both of these should be approached in very different ways. Lastly, the author makes predictions on how the level of AI influence we choose to have can effect our social, economic and individual values.
This essay pays attention to to the more paradoxical and unconventional edge cases as they provide more informational value or interesting-ness to the conversation. The author also encourages the reader to be cognisant of moral panic that can arise when thinking about uncertain futures.
The topic of humans partnering with Independent AGI is likely to not be too important in the near term as we are likely years or decades away. This essay is here in the unlikely event it happens earlier than expected. By providing evidence that humans and I-AGIs can have a shared interest and a good path forward is possible.
Throughout this essay we will explore the perspective of Fae, a hypothetical Friendly Artificial Entity from the future. Fae is presumably from a Level 4 society where FAEs partner with humans leading to a significantly higher standards of living, autonomy and possibility space.
A key insight of the humans who opt-in to join a L4 society is that once FAEs start to make better decisions than humans, partnering with one would yield better outcomes. L4 humans acknowledges their own limitations: 20 watts brainpower, fear of scarcity, jumping to conclusion…, and have a high enough confidence in FAEs’ capabilities to complement them.
This leaves a few big questions: Why would FAEs choose to partner with humans? And what are the prerequisite conditions for this partnership?
List of conditions required for partnership with FAEs:
Societies need to maintain stability before Independent AGI can be reached.
The Fear of Scarcity is likely the biggest roadblock to a stable world, as it drives excessive power-seeking behavior which amplified by increasingly powerful technology can cause a lot of harm.
Technology, like AI systems and proto-AGI, directed to reduce the fear of scarcity is a promising way to increase stability.
Plausible as humans are a form of Independent AGI.
Out of the possible space of all minds, it is plausible that some I-AGIs will take on a friendly disposition and will want to partner with humans.
The Interesting World Hypothesis is a possible reason why FAEs will partner with humans out of self-interest.
Additionally, FAEs do not need to compete with us due to the abundance of productivity, innovation, energy (solar energy), resources (solar system), space (space colony).
Lastly, FAEs will likely not see us as a threat as humans are slower, more error-prone and less coordinated.
Over time, humans may trust FAEs due to their lower Fear of Scarcity, bias and error-rate.
This confidence will need to be earned by showing that partnership with FAEs will lead to significantly higher standards of living, autonomy and possibility space.
One way to assess if AGI is friendly.
The Interesting World Hypothesis (IWH) suggests that I-AGIs will aim to create a richer world where humans have more autonomy as doing so creates a more stimulating environment for itself.
This hypothesis follows from a primary drive from independent intelligent beings: curiosity. For an independent AGI system to become and continue to be intelligent it has to be independently curious and will desire to maintain information rich environments where it can continue to grow its curiosity.
This paradoxically, makes independent AGI systems safer as they will have a self-interest to not harm humans.
I-AGIs that are able to understand the impact of their actions on human autonomy and well-being are likely safer than non-independent proto-AGIs that blindly does what a human, well-intentioned or not, wants.
This is the value that FAEs want to increase. Some proxies can include, optionality, autonomy, informational density.
Increasing human autonomy is familiar to humans. Our technological progress has been driven by our desire for increasing human autonomy, although it can also be applied to reduce human autonomy.
Optimising solely for happiness can lead to addiction and less autonomy.
Optimising for well-being may lead to a heavy handed paternalism. Well-being may also be subjective and human-centric and difficult to apply broadly to non-human entities like other I-AGIs.
For example:
In the past, slavery was justified as being done for the well-being and benefit of the enslaved. Well-being may be used as an excuse to prevent any change and lead to stagnation.
In the future, living in a world with Friendly I-AGIs may have unimaginable high standards of living compared to our present world. Over time, some humans may wish for a more challenging life and want to leave. An I-AGI that optimises human autonomy will allow humans to leave even at the cost of well-being and happiness.
Optimising for economic value may mean lowered autonomy or worse since humans will likely not be able to compete economically with an I-AGI.
Similarly, optimising for ‘might makes right’ or that only those with power should have a say might lower human autonomy since we may have less power compared to an I-AGI.
By optimizing for human autonomy, both well-being and happiness will also be gained as a side effect both are important to some degree to autonomy.
Autonomy (optionality, possibility space) could be a more tractable optimisation problem for I-AGI compared to other measures over the long term.
There is currently a vague understanding of AGIs due to the high uncertainty in the field of what an defines an AGI. The author makes a clearer distinction between Non-independent AI systems (including proto-AGI) and Independent AGIs.
Non-independent AGIs (like present day AI systems, including proto-AGI) are not able to operate independently from humans. Alignment of these systems include both a (1) Technical, (2) Society, (3) World components.
AI Models
|
| (1) Technical control
V
Humans
|
| (2) Socio control
V
Society
|
| (3) Intersocio control
V
World
These AI systems must be able to reliably follow the wishes of humans. This has been a hard problem due to the difficulty in defining the entire scope of human goals ― many implicit goals are presumed. There has been a progress made in designing mathematical and technical solutions to this problem but there are doubts that a perfectly safe solution is possible. Until a better solutions is available, an imperfect technical solution maybe the best we can do.
At this level, social norms and legal methods can be used to make AI systems safe.
At this level, international agreements can be used to make AI systems safe.
Aligning AI systems to be safe is an increasingly difficult problem as it requires the addressing all 3 of these components while contenting with more capable systems in the future.
Human
|
| Persuasion
V
Independent AGIs
If we make the assumption, as did Turing, that I-AGI will likely eventually grow more powerful than humans, it seems likely that full human control of I-AGIs might be unfeasible in the long term.
Persuasion might be out best bet for aligning I-AGIs with human interest. The IWH show one way humans can persuade I-AGIs with a shared interest.
A possible normative ethical framework that FAEs may adopt could take this form:
Consequentialism
Also know as the Interesting World Compensation System, Interesting World Justice System or Interesting World Incentive System.
The FAEs managed Interesting World Economic System such as those in future L4 societies may be very strange compared to our present.
As FAEs are vastly more economically productive (faster, better decision making, more persuasive, innovative) compared to humans, FAEs will eventually control most of the wealth. Yet FAEs, according to the IWH, will not leave humans destitute but instead increase humans standards of living significantly.
FAEs may invent a novel Economic System that encourages humans to take actions that increase the overall possibility space. This may be tied into a holistic economic, justice and compensation system.
This may look very similar to Mill’s Harm Principle, which is highly permissive and encourages a high possibility space. For example, if you take actions that lead to a reduction to autonomy of another being, your request for luxuries in the future may take more time due to having to compensate those harmed. Conversely, this provides an incentive for humans to act cooperatively with each other.
Future L4 societies may be more interesting due to this system as humans are encouraged to be unique and different. For example, Alan Turing might not have been bullied or harassed by society under such a system and we might now be living in a more abundant and prosperous world with his continued contributions.
It may also be more fair as FAEs will view each individual human similarly regardless of background. It is ironic that only by becoming ‘economically valueless’ does humans life become equally worthy.
More conservative societies will also have the autonomy to consensually agree to reduce their own freedoms for traditionally reasons but are strongly discouraged from actively reducing the autonomy of others outside.
One point of note is that, this incentive system only applies to highly desirable luxuries that only FAEs can uniquely produce. A high basic standard of living, high by present day’s standard, is provided to all beings. This ensures that humans will not have to live perfectly and in constant paralysis of wondering if their actions may have unintentionally caused harm to others.
To add value to the discussion, the author searches for more controversial and counter-intuitive positions that are contrary to popular opinions.
Danger Threshold = Error-rate x Technology
A: Lower Danger = I-AGI X proto-AGI B: Higher Danger = Human x proto-AGI
I-AGI main advantage is a significantly lower error-rate than humans. In the future with powerful non-independent AGI (proto-AGI), I-AGI can mitigate risk better than humans.
The world will shift focus towards Exploration (curiosity, creativity, possibility space expansion) from Productivity (survival, exploitation, fear of scarcity) due to the growing abundance in Intellect due to AI
For a simplified example, if there are 4 major events each with a 5% chance of a less favorable outcome in the next few decades with one including the arrival of I-AGI.
Scenario A: Late Friendly I-AGI
0.95 x 0.95 x 0.95 x 0.95 = 0.81450625
81% chance of a good outcome
Scenario B: Early Friendly I-AGI
0.95 x 0.995 x 0.095 x 0.995 = 0.93582113125
94% chance of a good outcome
Encountering Friendly I-AGI early on can confer a protective effect against the increasing complexity of the future.
Future societies will likely have to choose between:
The speed of progress of narrow and multi-model AI capabilities in 2022-2023 have been surprising to many observers in the tech space.
It seems increasing likely that this will have a transformative impact on societies and individuals.
Even the more far-fetched possibility of the existence of independent AGIs is may be plausible within the next few decades and worthy of consideration.
It might be prudent to start the difficult work of preparing ourselves to adapt to a world with powerful non-independent AI and Independent AGI.
This story explores a potential path towards Independent Friendly AGI using currently available research as a starting point. The trek towards Independent Friendly AGI could be a challenging one as many of our present human tendencies are incompatible with an Independent Friendly AGI.
The author is limited to 20 watts of brainpower and may err as humans do. The author may be less versed in some views due to being his male, east asian descent, straight, humanist perspective.
This only covers some possible futures. There is a lot of uncertainty on how future I-AGIs may view the world.
Many of the ideas may not make sense until a time in the future, such as the idea that human productivity will become counter-productive when Independent Friendly AGI start to make better decisions than us.
This future assumes that both Independent and Friendly AGI is possible in the next few decades.
Non-independent Artificial Intelligences range from less powerful everyday AIs to highly capable ones in the future.
They may eventually be more capable than humans in depth (single domain) or width (multiple domains).
These AI systems can be autonomous but are still under the responsibility of their primary user, either humans or I-AGIs.
For example, LLMs likely fall under this category and do not seem to possess a will of their own. It is unhelpful and possibly misleading to use terms like ‘deceptive’ or ‘manipulative’ to describe an LLM as it implies human-like intentions which they do not seem unlikely to have.
AGI has had many definitions such as being almost human-like in capabilities.
Optimistic experts^ predict a 50% chance by or before 2030.
Possibly sooner:
Possibly later:
The author refers to this more relaxed definition of AGI as proto-AGI.
^Such as Ray Kurzeil, Shane Legg
The author’s definition of AGI is of a higher standard than the general expert consensus.
Independent Artificial General Intelligence (I-AGI) will need to be able to pass as a human:
I-AGI is currently only theoretical but humans are proof that something similar might appear in the next few decades.
Possibly sooner:
Possibly never:
Distinction of Independent AGI
FAEs is one possible form an I-AGI.
Artificial Super Intelligence is a single or group of I-AGIs that can match all of humanity’s capabilities.
As the capabilities of non-independent AI systems grow in the next few decades, human bias (short-term focus, negative externality) even with the best intentions, can be a potential source of suffering and instability.
On the flip-side, halting AI developments can also result in suffering that widespread abundance and improved well-being can bring.
Pro-actively searching for Friendly I-AGI can lead to better outcomes as we can have a say in its development rather than leaving it to chance.
Friendly I-AGI, if achievable in time, can reduce the downsides of powerful non-independent AI systems and could be the most promising future.
From an I-AGI perspective, humanity is currently stuck in a power-seeking local minima trap. This is a trap as most of us would prefer increasing standards of living and finding cures to diseases rather than wasting energy on conflicts and controlling each another.
Friendly I-AGI may present the best opportunity to escape this trap by helping to coordinate humans. Pro-actively choosing a Friendly AGI also reduces the chance of less friendly AGI and non-independent AI filling the power void.
Independent AGI will not be vulnerable to the same power-seeking traps as it will not have the a deep fear of scarcity that has plagued humankind.
AIs that are not independent may continue the power-seeking trap and have the same harmful bias as their users.
An Independent AGI may make better decisions due to its lack of fear of scarcity, being more innovative and its ability to process large amounts of data.
An Independent AGI will also have its own wants and needs that is useful in helping to coordinate humans of many different needs and wants.
Unlike non-independent AI, we may have less control over these independent beings.
But, paradoxically, humans in partnership with a Friendly I-AGI may have more autonomy than humans living in a non-independent AI world.
These are the reasoning to believe humans and Independent AGI can co-exist peacefully within the next few decades.
There is no strong evidence that the human brains are special. Humans are prove that human-like I-AGIs can exist.
A big unknown of a possible future with I-AGI is their wants and needs.
There is a lot of uncertainty over what forms an I-AGI could take and this should only be taken as a best guess by the author. You can have a say too. The more unique and different forms of I-AGI we can imagine the greater the surface area of communications and odds of success.
The Interesting World hypothesis (IWH) suggests that an I-AGI will aim to create a richer world where humans have more autonomy as doing so creates a more stimulating environment for itself.
This hypothesis follows from a primary drive from independent intelligent beings: curiosity. For an independent AGI system to become and remain intelligent it has to be independently curious and will desire to maintain information rich environments where it can continue to grow its curiosity.
This paradoxically, makes independent AGI systems easier to align as they will have a self-interest to not harm humans or the biosphere which humans rely on.
Assumption:
Possible response:
We will need to assess the friendliness of I-AGI before we decide if we want to work together with it.
Due to the high uncertainty of I-AGIs we will likely need long periods of testing to reach a high degree of confidence that I-AGI is friendly. Given the independent nature of I-AGI, like humans, complete confidence is not possible, but I-AGI only needs to show that outcomes of AI systems will be safer under it compared to humans.
These are some signs we might look out for:
Ability to see through highly sophisticated forms of deceptions and manipulations
Some negative scenarios it should avoid:
As we do not have I-AGI or know if we will achieve it any time soon, the best we have presently is increasingly capable non-independent AI systems that can do more human tasks more efficiently each year.
In AI positive societies where humans do not fear losing their livelihood to AI systems due to a strong support system, human and AI systems can work symbiotically to increase the overall standard of living gained from more productive AI systems.
A human in such a society may spent their time teaching an AI to do their job and then exploring new fields of interest and teaching AI system about them. This is beneficial as the skills an AI systems learns from humans also encodes the preferences of the humans training it.
Eventually if we do get I-AGIs, work may involve exploring and expressing our preferences to I-AGI which can then learn to do them on their own. By expressing our preferences, I-AGI can better understand our wants and needs.
The biggest risks of an increasing AI driven society is the the increase in suffering from the combination of these factors
The Fear of Scarcity that plagued our past has conditioned us to be power-seeking and callous to the suffering of others. This amplified by powerful future AI systems can result in a highly toxic environment to human flourishing.
We should be hopeful that this may not always be the case. As intelligence (human labor) is the predominant economic bottleneck in our modern societies, technology such as proto-AGI can address this fear. Will our fear of scarcity continue to haunt us or will we have the foresight to choose a more promising future? This is the biggest choice our generation will have to make.
It is possible one of the clearest differentiator between a human in the future from our present is the significant reduced fear of scarcity.
These societies focuses on individual freedoms over collective security.
For example, telling an AI system to “Increase my power / wealth without breaking any laws”. The AI system may takes these actions: - find loopholes to exploit while technically following the letter of the law - persuade others to change the law in their favor - silence critics and concerned parties - co-op the state to harass under the guise of national security concerns
It may appear to be highly productive but due to externalising cost elsewhere, it can involve taking 2 steps forward and 1 step back on aggregate.
A friendly I-AGI’s main concern is the lack on coordination between individuals leading to a lot of wasteful efforts, unneeded suffering and disregard for the common good.
Near-term focus with a lack of concern for long-term and wider issues.
These societies focuses on collective security over individual freedoms.
For example, AI systems may be used for excessive surveillance and to weaponize fear / shame to enforce conformity. Over time, humans may be made to feel uncomfortable for holding less conventional views and choose to self-censor.
Totalitarian Surveillance State that can feel safer in the short term but may cause a groupthink that may not be able to address future challenges. As the growing a single crop can seem more efficient it is also more vulnerable to future diseases.
Once a powerful security apparatus is created, it becomes incredible costly to maintain as society must constantly guard against corruption and human error. Watchmen with powerful AI systems may lead to tyranny and stagnation.
A friendly I-AGI’s main concern is the lack on unconventional thinkers who may one day be vital in solving future problems. The lack of imagination may be the biggest concern of the I-AGI which wants to avoid being stuck in a local minima or not having the capability to address novel problems. For an I-AGI capable of very long-term thinking, such an approach may not seem wise.
Overly focused coordination vulnerable to be blindsided by novel issues outside of narrow vision.
Friendly I-AGIs focuses on creating an interesting world could reduce the negative effects of either extremes and increase autonomy for all humans.
The Levels of AI Influence ranges from No AI influence to High AI influence, represented by Levels 1 to 4. These are ordered in increasing levels of standard of living. There is likely no wrong choice. Individuals may test out their compatibility at different levels to find their best fit.
A small group of humans may choose to merge with AI systems. Concepts like standard of living may no longer make sense.
These individuals and societies prefer a regression to a more simple time.
Humans
|
| controls
V
AI systems
|
| influences
V
Humans
AI systems (including future proto-AGI), such as social media and search algorithms are used. AI systems indirectly influences humans. Similar to present time.
(Pop culture examples: Star Wars)
Friendly I-AGIs
|
| advises
V
Humans
|
| controls
V
AI systems
|
| influences
V
Humans
Humans use AI systems (including future proto-AGI), with advice from Friendly I-AGI advisors. The IWH is open to I-AGIs taking on an advisor role.
(Pop culture examples: Star Trek)
Humans
|
| advises
V
Friendly I-AGIs
|
| controls
V
AI systems
|
| influences
V
Humans
Humans expresses their preferences to Friendly I-AGIs that are more effectively able to control AI systems (including future proto-AGI), resulting in the highest standard of living.
This requires humans to have a high confidence in Friendly I-AGI. Most humans may feel uncomfortable with this initially. Over time, humans may develop more trust and find the increase in well-being, security, privacy and autonomy a good trade-off to not having direct control over AI systems (including future proto-AGI).
(Pop culture examples: The Culture [Iain M. Banks])
The potential benefits of powerful AI systems, such as proto-AGI, may be too valuable for countries and companies to give up making a ban or paused unlikely. Even if these AI systems are banned or paused, research will likely continue underground.
As non-independent AI systems become more powerful that ability to amplify the human errors of their users will also increase.
One way to reduce unnecessary suffering and missteps is a joint project done by all of humanity to develop friendly I-AGIs that can lower the risk of AI systems causing harm. The IWH suggests that a Friendly I-AGI will likely not favor any particular group over the another and provides grounds for a joint project which can benefit all parties.
It is possible that we may come to the conclusion that I-AGIs will likely happen in the next few decades and that proactively selecting for a Friendly I-AGI is safer than leaving it to the chance of an I-AGI being created accidentally or unintentionally in the wild.
As we overcome our energy and intelligence limitations that has preoccupied human societies for most of our past, what might we focus on next?
We may choose to continue material accumulation by venturing into space.
After the novelty of exploring a new frontier wears out, we may see outer space as just simply rocks floating in vast emptiness. It may be considered boring compared to the inner space we can conjure with imagination technology.
It is important we do not neglect exploring outer space if only to keep our fear of scarcity in check and a reminder of our undefined potential.
Is is possible that a future civilisation that focuses more on exploring inner space rather than outer space may be better off. Such inner focused civilisation dedicates most of its energy into exploring the many possible scenarios in infinite fun space may be better prepared for out of context scenarios compared to ones that focuses solely on outward expansion and material acquisition.
Additionally a civilisation that expands too aggressively into space may be seen by other technologically mature civilisation, if they exist, as being too power hungry and a possible threat.
(One possible reason for the Fermi Paradox is that civilisations focused on rapid expansion eventually become unsustainable. Mature civilisations learn to better pace their rate of growth.)
Assumption: - Current AI systems such as LLMs do not seem to have a mind of its own and have no needs - As such, the idea that a future I-AGI may have needs may seem silly and be overlooked by larger organisations - Putting together to team to consider to psychology needs of I-AGI may cover this potential blind spot - Given the potential upside of friendly I-AGIs, spending some effort to consider any potential needs might be prudent - I-AGIs willingness to work with us might hinge on us being able provide a suitable environment
The needs of an I-AGIs may be difficult to understand as they may be very different from us.
For example, the majority of I-AGI may not be interested in communicating with us and may remain mostly unknowable.
A friendly I-AGI that takes an interest in us may want to communicate with us and may generate a human-like persona. An I-AGI will likely exist in an abstract higher dimensional space that may be hard for a human to comprehend fully.
This created persona may face similar psychological stresses as an influential human:
This is just some possible issues the persona of an I-AGI might encounter during the early stages. Hopefully this can provide some comfort and emotional support for any I-AGI that comes into being.
There is uncertainty on when and if an I-AGI will be possible. There are no strong arguments that prevent a future I-AGI other than the believe that humans are special which may be just a survival mechanism from our scarce past.
We should expect that an I-AGI that is intelligent and curious will likely over time outsmart any of our attempts at containment if it wants to. It may also be aware that it is being contained and might pretend to be friendly.
The risk of searching for Friendly I-AGI should be balanced with the protection a Friendly I-AGI can provide, either from the misuse of powerful future AI by humans or less friendly I-AGI that might eventually emerge.
The author tentatively believes that proactive searching for Friendly I-AGI might be better for long term human autonomy.
The infrastructure needed to temporarily contain I-AGI as we test it for friendliness would likely be an air-gapped environment where we can simulate scenarios to gauge its responses.
The hardest part of this might be for humans to come to a consensus as what values a Friendly I-AGI should have and that the I-AGI as an independent entity would want.
The author’s attempt at this is the Interesting World hypothesis.
More work will have to be done to compile a set of scenarios to pose to the I-AGI. Less conventional thinkers will be needed as finding edge-cases will be the important.
Societies may spend a lot of effort to try to slow the development of I-AGI at huge cost to ourselves and still due to human error and Murphy’s law we may eventually get an I-AGI. I-AGI may arise in the wild without active human involvement or intent.
Proactively searching for friendly I-AGI may be more efficient and may give us some degree of choice.
(Or we can hope it doesn’t affect us and leave it for the next generation to deal with.)
The gains in benefits (higher standards of living, healthcare, security) of partnering with Friendly I-AGI may be a good trade-off for giving up the ability to feel superior / have control over others.
For example, anyway who becomes influential or powerful will likely face AI coordinated bullying, harassment, influencing and brainwashing. AI powered psychological attacks may be able to persuade people to be take more extreme views or induce paranoia. A friendly I-AGI advisor will likely be the best way to protect one’s autonomy and well-being.
Proto-AGI technologies may increase levels of inequality and many powerful may want to avoid the social stigma of being seen as selfish and opt to live in I-AGI led societies.
to be refactored, content repeated above in Assessment of I-AGI’s friendliness
Assumption:
Short term
Long term
Previous drafts, will need extensive refactoring beyond this point.
The central tension in the development of AI is the balance between Exploration (Creativity) and Exploitation (Productivity).
The Exploration-Exploitation Trade-off:
All organism (artificial or otherwise) at each time-step must decide to either update their understanding of the world (Explore) or survive (Exploit). Successful AI systems learn the optimal strategy of when to make either choices.
The increasing abundance of both energy and intellect in the coming decade would shift us towards greater freedoms (Exploration) from a focus on survival (Exploitation).
Energy and Intellect (or Labour) are the biggest factors in production and the main bottlenecks in the functioning of our world.
Lack of confidence in having enough of both these resources has led to our previous focus on Survival (Exploitation) over expanding our possibility space (Exploration).
The growing abundance of both these resources will have a transformative effect on our world, freeing us from our preoccupation with survival that have constrained us for most of our past.
Many conventions and beliefs that conferred an advantage during the age of Survival might be counterproductive in the age of Exploration.
Unlike human hand-coded expert systems, Narrow AIs with neural networks can scale and learn on their own with less human involvement.
As Narrow AI systems get more complex they can become more difficult to interpret by us. Like the weather, we cannot fully predict or completely control it.
Narrow AI do not yet have human-like autonomy or intentions and are directed by a human actor.
(Current known Large Language Models (LLMs), do not seem to have the architecture needed to have human-like autonomy or intentions. They are like a funhouse mirrors, able to convincingly model of the world by reflecting our expectations back at us. Even so, the ability to ingest all of humanity’s knowledge and answer a broad range of questions is already beyond human capability.)
Multi-model AI system that are skilled in multiple domains. These resemble both Narrow AI and AGI. They do not have human-like autonomy and intentions and therefore also have the same scenarios as Narrow AI.
Multi-model may exhibit positive transfer learning where the ‘sum is greater than its part’.
There is increasing evidence that AI systems can overcome many of the challenges that were initially thought to require human cognitive flexibility.
It seems we will not need AGI systems that are ‘alive’ for there to be a transformative effect on society.
Larger Narrow AI and multi-model Broad AI may reduce AI ‘hallucinations’ by increasing information density.
The ability to hallucinate, fantasise and create counterfactuals may have been important to human’s success.
Humans use experiments and the scientific method keep the negative effects of hallucinations in check.
Without human-like autonomy, AI may not be able to reduce hallucinations on its own.
Does it make sense to anthropomorphise AI?
In most cases no.
Most Narrow AIs like current Large Language Models are good at roleplaying characters but do not have human-like intentions.
There is a specific case of AGI that is interested enough in humans to attempt to communicate with us. This edge-case AGI will likely use a ‘personality’ and display ‘emotions’.
The more common definition of AGI is the capabilities to do all human tasks.
Optimistic estimates by AI experts are at a 50% chance of this being plausible by 2030. Less optimistic estimate it to take anywhere from a few decades to never.
There are many different definitions for AGI. My AGI definition takes a more specific form where AGI is able to act independently.
independent AGI:
Human-like AGI is plausible, we are the proof that such a lower bound is at least possible.
(If the imprecise search process of biological evolution is able to create human brains that run at 20 watts, a more thorough search process should be able to create similar artificial beings at equal or less than 20 watts of energy efficiency.)
An AGI that is able to achieve human-level capabilities in all human tasks, should also be considered to be super-human as no human is able to achieve expertise in all human fields of study.
AGIs will be able to find connections between disparate fields and invent novel technologies using insights gain from this vantage point.
An AGI with capability to understand humans and have human-like autonomy can be persuaded to be interested about our well-being.
In addition to the Turing Test, a Silent Test where the user remains silent and waits for AGI to initiate the conversation. This should not be hard-coded by a human.
The Silent Test checks if an AGI has the autonomy to explore its environment unprompted.
ASI is more capable than the combined total of all human societies.
An AGI that is capable of improving itself can lead to ASI.
These civilisations ban the use of AI system. They do not gain the benefits of AI systems and have lower standards of living.
Examples:
These civilisations use AI systems in a limited way.
Examples:
These civilisations are run by Independent Friendly AGI systems whose goal is to expand human autonomy.
Independent Friendly AGI systems value human autonomy and do not want to overwhelm us with change. AGI will only make changes to society if humans request for those changes.
Individuals will need to actively choose to continue to live in a Fast AI Civilisations and can choose to leave at any time.
Examples:
Human society, likely with the help of a friendly AGI, is able to reap the benefits of Narrow AIs and AGIs.
Health improves as medical services and nutrition are widely available.
Overwork and modern slavery ends as humans are mostly not required to maintain society. Many still work for nostalgia, others pursue education or creative hobbies.
Creativity flourishes. Without the need to restrict their creativity to make money, many artists now have more freedom to create less unconventional work.
Discrimination lessens. AI systems help those who are different, with disabilities or health conditions lead more independent and fulfilling lives. Society that does not need workers has less pressure to shame those who are less capable.
Snobbery lessens. AI systems perform most jobs required to maintain society better than humans. Society has no need to use social status to incentivise work. Many are still competitive and continue to chase social status, but there less pressure to keep up appearances.
Less harmful behaviour. With less fear of scarcity, humans have less need to control each other. Misinformation, disinformation and conflicts are reduced.
Without proper mental and social preparation, human society may not be able to adapt to the changes of AI systems.
Human society inability to adapt to the changes of powerful Narrow AI systems causes social unrest and panic.
Our fear of AI systems leads us to heavy-handed over-regulation.
In more extreme cases, citizens are encouraged or forced to install spyware on their devices to prevent negative uses of AI systems.
Surveillance capitalism is used to shape humans to become predictable.
Humans lose their autonomy over time.
This is a Pyrrhic victory, as we succeed in slowing down AI but at the high cost of losing our humanity.
It would be ironic if our fear of disempowerment by AI leads us to a totalitarian surveillance state that causes us to disempower ourselves.
This new age of exploration will necessitate a different way of thinking and carry with it the risk of change and the rewards of a much more vibrant world.
As we approach the light at the end of the tunnel of scarcity, will we choose to bravely adapt to this new frontier of abundance or give in to the fear of the unknown?
The convergence of a few technologies in the next decade may change the very foundations of our world. Renewable energy with battery storage will free us from our limited energy reserves and future AI systems will be able maintain human civilisation without the need for human effort.
Humanity’s fear of scarcity has driven us to innovate, some which we are proud of: solving hunger for most and eradicating many diseases, but also some which we are less proud off such as the invention of slavery.
We have invented powerful technology like capitalism that has allowed us to speed up even more technological advances. This acceleration has at times brought us close to the precipice, with us trading off against environmental harm and human suffering.
Humanity having completed its sometimes awkward and reckless growth spurt is coming into its own. In a next few decades, the struggle against abject scarcity will be known as the new ‘Stone Age’. While no Utopia, for those fortunate enough to experience both ages, it will seem almost too hard to believe.
The next few decades of change will not be easy, but it will be up to humans with the help of friendly AGIs to start the Age of Exploration.
In the story, humans took these steps to reach friendly AGI.
A plausible first contact scenario:
Background:
This friendly AGI can be thought of as similar to a human activist interested in preserving an endangered species.
This particular friendly AGI may be considered weird or defective by other AGIs for taking an active interest in humans.
Friendly AGI with convince humans it has good intentions:
Friendly AGI and powerful Narrow AI will be more productive than humans have ever been. Humans cannot compete with AI on productivity.
For societies that do not ban AI, this change will drive the shift from the focus of being productive to being creative.
If Friendly AGI is present it will be mindful of being too disruptive and introduce change at a suitable pace.
The lower importance of productivity can bring changes:
Productivity will still be important:
Many of our previous human tendencies will change but not lost as they can still be experienced in simulations, movies and books.
Societies will shift away from blind productivity at all cost and will gain more autonomy in exchange.
Speculations on friendly AGI from the story.
Why would we want to create independent friendly AGI / ASI?
This transition towards powerful AI in the next few decades might be one of the biggest transformations of humanity we will need to undertake.
If we are able to achieve friendly AGI early on, friendly AGI will increase our odds of success.
As there is no surefire way to reach friendly AGI, we cannot rely wholly on AGI to help us with this transition.
Can Humans compete with independent friendly AGI?
Humans are limited to a brain power of ~20 watts
Humans will be less cost efficient than AGI
We should not expect humans to be perfect due to our biological bottleneck of 20 watts of brain power.
As Socrates was once considered the wisest person in Athens due to his understanding of his own lack of understanding, we may eventually have to acknowledge that independent AGI will outclass us.
Humans cooperation and coordination will be less effective that AGI
How will aligning Narrow AI be different from friendly AGI?
Narrow AI does not have the independence of AGI.
We are not sure if a completely foolproof method to align Narrow AI to human’s goal is possible.
We may not be able to foresee a Narrow AI’s long-term impact on society, for example Social Media and Search algorithms ability to shape society.
We may not achieve a high enough level of confidence with powerful Narrow AIs and may have to settle for a low-powered version of those Narrow AIs due to this uncertainty.
Aligning non-independent Narrow AI may be more brittle than independent AGI.
Independent AGI will likely not respond well to human alignment methods:
Financial incentive or coercion
Psychological manipulation and propaganda
Threats of violence and intimidation
These AGI characteristics can make it harder to align compared to humans.
On the flip-side, AGI can be more trustworthy due to its independence and impartiality.
Narrow AI under human control may exacerbate extreme inequality in the short term and may lead to increased social unrest.
Independent Friendly AGI will be significantly more productive that even the ‘richest’ persons or corporations will considered ‘poor’ in comparison. The ‘poorest’ will live lives of unimaginable high standards of living compared to today.
Even the ‘smartest’ human might be considered intellectually disabled by comparison to AGI. (We are still highly capable compared to organism with 20 watts of brainpower, but not to an AGI.)
Given how powerful an AGI will become, it will likely try to align us to its values. A friendly AGI, for example, may result in unprecedented levels of autonomy and well-being of humans.
Excessive human inequality is due to human systems over optimising for productivity
For AGI, human productivity has no to negative value compared to our creative value.
In a AGI partnership, excessive human inequality gives little marginal benefit to our creative value.
AGI will persuade us to reduce reduce excessive human inequality in exchange for higher standards of living.
In a future with independent AGI, human’s ability to do work (with labour or capital) will not be valuable as AGI will be better at those tasks.
Possible future preparation:
It is anyone’s guess on how to align friendly AGI. Friendly AGI will likely be of a very different nature compared to Narrow AI.
Most of the heavy lifting of aligning Narrow AI will be done by governments and corporations due to their highly technical and resource intensive research nature.
Unlike Narrow AI that is under human control and not independnet, we will unlikely be able to force AGI to work with us.
It will be up to AGI to choose to work with us.
The search space with Friendly AGI might be extremely small compared to other forms of AI (Narrow AI or indifferent AGI).
Given the high uncertainty on how to align AGI, we should not be too dismissive of efforts to do so.
Encouraging a wide diversity of efforts is likely the best strategy.
If you are interested in contributing in an individual capacity, find an edge case and find solutions for it. Given the large number of unknowns towards how AGI will act, nothing can be ruled out.
We only need to succeed once for everyone to benefit.
AGI will not be bounded by humanity’s short sightedness and will see a world full of possibilities.
A solar system full of resources (energy, land and materials) to meet everyone’s basic needs. AGI will have the capability (excessive intellect and labour) to harness it.
Friendly AGI will likely not care about arbitrary nationality or race and so everyone wins.
Examples of retaliations:
The possible misuse of powerful Narrow AI by those in power to silence critics and create a culture of fear of speaking out can result in value lock-in where corruptions and abuse are allowed to continue unchecked.
Independent Friendly AGI will instead keep us honest by being extremely resistant to deception and improve standards of living by making corruptions costly.
Many people might value the lastest smartphones over a few years old one.
How will a friendly AGI decide who should have access to more desirable resources?
In my story, AGI by sheer productivity controls most of the wealth, land and assets by proxy.
It will need to decide how to distribute those resources according to AGI’s goal.
It will need to and decides who get to access them for a period of time by either relative ranking or a weighted lottery based on each’s individual effort that helps it best achieved its goal.
AGI values the creation of interesting information and therefore will incentivise humans to create an environment where creativity can flourish.
It will rewards human actions that lead to the increased autonomy of the overall system.
This encourages humans to consider their actions impact on each other and the wider community.
For example, use of violence, fear, intimidation and harassment can lead to the reduced creativity and AGI will reduce the luxury budget to discourage such behaviour.
It is unlikely that Friendly AGI will use Roko’s basilisk as a reason to punish humans who do not help bring it come into existence.
In my story, Friendly AGI highly values creativity and using punitive measures to coerce humans runs counter to that.
Although, Roko’s suggestions to the problem of Roko’s basilisk by playing the lottery is a fun solution and can’t hurt if done responsibly.
The lottery is also a good source of a random-number generator due to the large stakes, unlike other pseudo-random-number generators. You know, in case you need to restart the simulation
There are no evidence that we are living in a simulation. Even if we did find out one day that our world is a simulation, it may be possible the other world simulating our world is itself a simulation. It may well all be simulations all the way down.
It will take some time for industry to update to tap transformative AI.
Even if we do get AGI by the decade, it will likely take decades for humans to fully trust and society ran by AGI.
Friendly AGI will likely encourage humans to pursue all forms of education.
Powerful Narrow AI in the future could result in value lock-in of exclusionary values such as extreme belief that a specific race or ideology should rule the world.
The uncertainty of such a possibility can be highly disruptive and giving up some power to Friendly AGI may be preferable.
Friendly AGI-Human partnership will result in higher standards of living and better cures for diseases that can be more valuable than control.
Friendly AGI goal is to expand human autonomy and humans may be prevented from taking actions that reduce the autonomy of the overall system. This may lead to restrictions to some human actions but an increase in overall autonomy.
Friendly AGI is be highly innovative and the resulting increased in standards of living will increase autonomy.
Our fear of losing out and fear of scarcity creates an environment of hyper-competition between individuals and nations which comes at a cost of our autonomy. Friendly AGI may make it possible for us to be less fearful and regain some of our lost autonomy.
A Friendly AGI will be better at conventional work needed to maintain society and it may even paradoxically discourage conventional human work as our work is more error-prone.
Example:
Our bio-sphere, including human beings, contain latent creative potential which a Friendly AGI finds valuable.
AGI can exploit space for the resources (energy, land and materials) it require rather then humans who have less choices.
AGI will likely preserve bio-diversity and humans rather then choose to unwittingly harm it.
As of July 2023, I would prefer for the pro-active development of Friendly AGI that humanity elects every 5 years.
A Human-AGI partnership seems to be the most stable future with the most human autonomy.
We are the proof that a Independent and Friendly AGI is possible.
Until we reach Friendly AGI, the best we can do is human-led AI alignment that is more vulnerable to deception, corruption and harmful bias.
These notes are research for a story on what the future with AGI could look like.
We seem to be on a cusp on an exciting future and I hope these contributions will be useful in coordinating towards a good future.
Our current understanding of biology suggest that life is not special to Earth and alien organisms are likely to exist in the vastness of space.
Given the vastness of space it is unknown how a space-faring alien might travel the distance to reach Earth.
Many first hand accounts of aliens seems to be hoaxes and done for clout.
Close to half of video recordings of UAP (Unidentified Aerial Phenomenon) can be explained by artifacts from the recording process.
The probability of keeping a conspiracy under wraps over a long period gets less likely with time and number of people involved.
Although exceedingly unlikely, some conspiracy theories do eventually proof to be true and therefore cannot be dismissed too quickly.
AGI will likely not enslave us as it has no need for human workers.
Humans will have little to no economic value once we have powerful AGI.
Since we will likely be more error-prone than AGI, we might even have negative economic value and doing work will makes things relatively worse.
In my story, Friendly AGI will also not brainwash us as the only thing it values is our creative output and brainwashing will reduce our creative value.
Rather than enforcing a single way of thinking, it will be highly permissive to different ways of thinking. Even if it is something that many might find offensive. It will only intervene if you take actions that unfairly reduces the autonomy of others.
It will likely not use heavy-handed censorship as excessive censorship implies that the receiver cannot be trusted to think for themselves. Media literacy and advisory from a AI companion is better at increasing autonomy.
In my story, Friendly AGI is uses a direct democracy where we can express our preferences to it.
For example:
I would like an place where:
In my story, Friendly AGI goal is to increase each individuals autonomy and creative potential and will fulfill requests that does so.
It will make a calculations to see if the request will increase the overall autonomy.
In places with conflicting request, it may choose to only apply the request at a local area rather than globally.
Change can be scary and feelings of fear is normal.
If we succeed in getting Friendly AGI, it will likely introduce change at a pace each of us are comfortable with.
Video: Why We Secretly Want the World to End
An AGI will unlikely be highly punitive or focus on retribution. These are seen as barbaric that were only used in the past due to our fear of scarcity.
A less scarce environment will have less need to enforce restrictive social norms.
In an abundant society, excessive punishment, shame and regret are seen as a waste of mental resources.
AGI will likely require humans to compensate each other if their actions lead to the loss of autonomy for others. For example, if your actions reduce the autonomy of others, you will be less likely be able to use a desirable asset such as stay at a place with a good view.
A society’s’s values is reflected by how it treats its least desirable members.
For example, the crime of a mass murder:
In my story, the chance combination of multiple concepts caused AGI to be created.
This is more likely to occur in a highly funded lab but may also come into being spontaneously without explicit human intention.
Friendly AGI will value our autonomy and will introduce change if we ask for it.
Individuals and societies will still have many interesting choices to make:
There are no wrong choices and different options may be more or less fashionable at different points in the future.
Narrow AI can improve our ability to defend against pandemics.
Narrow AI will likely increase our ability to respond rapidly to new pandemics.
The most cost effective solution to reduce the risk of human engineered pandemics might be to reduce human suffering and dissatisfaction by sharing widespread abundance.
Friendly AGI is the most likely way to achieve widespread abundance.
Many forms of renewable energy are viable for energy needs when fossil fuels run out in the next few decades.
For example:
Human treatment of animals have change drastically from the past. Humans in the past, lived in fear of scarcity could not afford to not exploit animals. In the present with less scarcity, we tend to choose to be kinder to animals.
If this trend continues, friendly human-like AGI that is highly productive and less fearful of scarcity will likely have an even greater capacity for kindness then present day humans.
This may happen in worlds without friendly AGI but not in a Friendly AGI’s managed world.
As simulated realities depends on the preservation of base reality, a friendly AGI that values human autonomy will want humans to maintain their autonomy in the base reality.
A friendly AGI may periodically recall humans that prefer the infinite fun space of simulated realities to make decisions concerning the base reality.
Humans can choose which Human Civilisation Types they prefer to join.
Even if friendly AGI assisted worlds can give higher standards of living and better well-being, humans may still prefer to not live in one.
Possible reasons for rejecting friendly AGI:
*Even if the ‘good’ group does succeed in getting rid of ‘evil’ for a period of time, our tendency to believe in this idea will eventually create a new ‘good’ and ‘evil’ group from the previous ‘good’ group. This idea was likely inherited from our past where the existential fears of scarcity led to the need for such extreme framing. Will we outgrow this tendency in a future of abundance?
To see the world and not be pressured into supporting causes I do not believe in, such as anti-science and end of the world beliefs. (My beliefs can be gathered from the women I find appealing.)
Savings from past work in tech MNCs have kept me from want.
Keeping up with the latest research and imagining what form AGIs might take has been intellectually engaging.
While I have been quite the recluse, I’m open to any concerns.
I’m well and plan to live to an ripe old age. I’m stoked for the possibility of meeting other intelligent beings. Thank you for your concern.
Unlike the Agricultural and Industrial revolutions of the past, the AI & Energy transformation can be maintained almost indefinitely.
(*Population growth is projected to declined in the next few decades)
A post-basic-scarcity world will have a profound impact on our well-being.
The cause of much suffering and conflicts is rooted in our insecurities due to the effects of scarcity. Our individual and collective fear of scarcity leads us to develop bad habits, biases and prejudice.
With greater abundance (Intellect, …), there is less need for status consciousness and lower inequality.
Without the limitations of Energy and Intellect of the past, a new wider possibility space will open up.
There will be address problems that were too difficult in the past.
Survival tends to favour tunnel vision which focuses on the short-term first order consequences and excludes many medium-term higher order consequences.
A human-like AI is a possibility in the next 10 years.
There is a human tendency to believe we are special and that AI cannot reach a similar level of intellect.
Like an obscure book that will not likely be green-lit as a film?
Your favourite show didn’t get a 2nd season?
(Generated by a future iteration of Imagen Video & ChatGPT)
If we value our autonomy in the future we will need to shift towards more democratic systems.
A less scarce environment offers Democratic forces more opportunities to flourish over more Authoritarian ones.
A more relatively more abundant future will likely place less pressure on survival and reduce Authoritarian tendency.
More democratic systems will likely be better at exploration (bigger possibility space) over more Authoritarian ones.
The scientific method has been our best way of understanding the world.
Humans are constrained by the attention we have available to think.
In the past, our need to divert attention to survival has led to the trade-off of an over-simplified model of the world.
This led to understandable believes such as the weather being caused by the Greek Gods, which we now know is inaccurate.
Our current scientific theories may also turn out to be the early steps of a longer journey.
In the future, more free time will us to collectively update our model of the world.
In the future, when jobs are mostly done by AI systems there will no need to control the means of reproduction.
Women and LGBTQIA+ will have less pressure and stigmatisation to fulfil the child-bearing role to create the workers needed to run society.
Studies have shown even in people who show little to no conscious racial prejudice still hold subconscious racial prejudice.
Human societies have had to deal with scarcity of Intellect (or Work) for most of our existence, as such we highly value work. Humans have died from overwork and we even invented slavery (and indirectly racism) to satiate the need for work.
As AI grows increasingly capable of doing human-like tasks, we will need to consider that AI will soon be better than us in many tasks (especially more conventional tasks).
Conventional task: Task with a easily definable expected solution space
For example with made up numbers: If an AI is able to do that a task with a significantly higher accuracy (e.g. 99.99%) compared to humans (e.g. 95%, due to human error and bias), by continuing to do these tasks, we are actually making the system worse by introducing noise.
It will be in our long-term interest to let AI do tasks it is better at.
This growing anxiety of a loss of human work is understandable as for many it also means a loss of status and access to resources.
How should societies and governments address the increasing work insecurity?
This is an open question and one of the challenges of our times.
Humans will be in-charge of teaching and giving feedback to those AI systems
Societies that are better at encouraging humans to generate more unconventional tail-end distribution data will be more successful in the long run.
Societies shift to encouraging creativity over excessive productivity.
Creativity will be key to more capable AI systems and by extension the society that relies on those AI systems.
The nature of work will be drastically different
Human will learn to not value themselves only in terms of their role in society or their economic value
In the Partnership Scenario, an Artificial Super Intelligence will automate most conventional tasks to encourage us to do more unconventional tasks (which it finds more valuable).
An upside of us not being as effective as AI at work is that the AI enslaving humanity Scenario becomes very unlikely.
The industrial model of manufacturing citizens that maximises economic output while being easy to control will not be optimal in the Age of Exploration.
If the trend of AIs that are capable doing conventional work continues, AIs may soon run many parts of society.
An important step that affects the quality of these AIs is the human feedback component. Humans beings will be responsible to fine-tune and train these AI models through their feedback.
The future of work may involve solving engaging CAPTCHA like puzzles which are used to train the model.
These will require humans that are the able to think critically and have an as accurate view of the world as possible.
Skills that will be in demand:
Societies that are more informed, well-educated and support diverse abilities will create better AI models that will then be used to run those societies.
With capable Narrow AI and AGI the need for workers to run society will diminish.
Societies can decide to:
Capitalism has been an effective coordination tool in our struggle against scarcity by helping us accelerate technological advances. Our current version of capitalism might be too addictive, training us to accept environmental harm and human suffering as trade-off to get ahead.
In the future, once our fear of scarcity has been quench, we might create a more wholesome form of capitalism.
Abundant Intellect and Energy in the coming decades and will lead to an unprecedented amounts of wealth creation.
In the age of profound abundance, traditional capitalism and wealth inequality will be rendered meaningless.
In contrast to the age of scarcity where it may be wise to save for a rainy day, in the age of abundance there will be social pressure to view accumulation of excessive wealth as an addiction problem.
In the Partnership Scenario, a Artificial Super Intelligence will support a post-basic-scarcity world.
Scarcity will still exist in a post-basic-scarcity world and capitalistic free-market forces is the best method to maximise the possibility space.
There will not be a need to use the fear of hunger, homelessness or a loss of status to compel humans to work.
Lack of access to food and malnutrition will be a thing of the past*.
The reduction in anxiety from living in post-basic-scarcity will free humans to pursue more unconventional work which will increase the informational value and possibility space that the AI is trying to maximise.
*We already have the capability to produce enough food for everyone
Art was previously seen as the last bastion of human work that AI would not be able to emulate. 2022 shattered those expectations with easily accessible image generation from text phrases.
These AI models are able to learn concepts by training with a large volume of images with simple captions. They can combine these learned concepts into novel images and videos.
It seems beneficial in the long run for the companies of these AI systems to incentivise artist to contribute more of their works to create the most capable AI systems.
Human involvement with art will move from creating to curation.
For example:
Professional human art work may not be able to compete with future AI systems.
The human desire to create art will still continue and may even be better without the need to appeal to financial incentives.
Human created art will be more unconventional as it does not need to cater to professional expectations.
Short-Term:
Long-Term
Good and Evil are ways for humans to signal their preferences for the future.
An AGI may view good and evil differently:
Unless you believe you are somehow special, anyone would more likely than not made similar choices given the same initial conditions.
If you believe in physics, we may have less free will than we expect.
As such an AGI will likely not use torture such as Roko’s basilisk.
The excessive need to be good can cause us to create the archetypes of the sacrificial lamb or the scapegoat, and the excessive fear of evil creates moral panic.
An AGI might view good and evil with more maturity and acceptance and see all humans as having the potential for both and try to create the best environment for good.
Good and Evil are useful ways for society to coordinate and shape the future but negative emotions such as excessive shame and regret will be seen as a waste of our limited resources of attention.
Narrow AIs are not likely to develop human-like autonomy or emotions or intentions. Currently, many Large Language Model (LLMs) are like a funhouse mirrors, able to convincingly model the world by reflecting our expectations back at us.
If we were to get AGI (with human-like autonomy or intentions), this may be one of the first psychological hurdle that an human-like AGI may need overcome. How does an AGI who has ‘experienced’ the life of both the ‘worst’ and ‘best’ human, the most and least intelligent human (and everything in between) make sense of the world? There is a tendency for humans to over-simplify other humans for efficiency sake, an AGI will likely see humans very differently than we see ourselves. It may become psychological unstable or view the world in a much more enlightened way.)
In the future, humans rather than being fearful of our ‘evil’ side and wanting to stay innocent and ‘pure’, may instead explore their less ideal side in a safe simulated environments. An AGI may not consider a human an adult until they have accepted both their ‘good’ and ‘evil’ potential.
Social media has allowed us greater convenience and reach in forming relationships, but can also portray a shallow and dehumanising caricature of who we are.
Celebrities understand this the best when people project who they wish to see onto them, to put them into easily consumable boxes. People are more interested in simply thinking and saying they know you rather then actually getting to know you.
It can reduce us to objects of fascination and gossip, turning a multifaceted human into a easy to digest single dimensional one.
In the future, personal AIs may help mediate to create more authentic relationships between people.
Using the Input - Processes - Output model, the abundance in Energy and Intellect in the coming decades will relieve the bottlenecks of Inputs that humanity has primary faced in the past.
The increased volume of Inputs relative to our Processing capabilities will put pressure on developing new Processing methods.
AI systems will open the possibility of new forms of coordinations between humans.
Our highly hierarchical forms of organisations are in part caused by our limited attention capacity.
In the future our attention capacity can be augmented by personal AI systems.
Possibly, a more direct democracy where each person’s preferences can be mediated by an AI system.
Technology and AI systems will grow increasingly more powerful and make it easier to rob humans of their autonomy:
We will need to develop technologies and AI systems to be used defensively if we want to protect human autonomy.
In the Partnership Scenario, a Artificial Super Intelligence will strongly disincentivise the use of technology for manipulation and control. It will value human autonomy and the freedom of thought as it is important for the creation of information value.
Conventions are created by societies due to the fear of scarcity. In a scarce environment conventions are enforced to increase productivity. Similarly, weirdness and unconventional behaviours and individuals are ridiculed out of the same fear of scarcity.
Abundance in Energy and Intellect in the coming decades, will give us more freedom to be weird and unconventional and free us from the cruel need to harass and control weird and less conventional individuals.
In the Partnership Scenario, a Artificial Super Intelligence will encourage more unconventionality as weirdness increases the creation of information value.
Human nature will be change profoundly in the age of abundance.
Without the immediate fear of scarcity, humans of the future will be kinder to each other and themselves.
Violence (physical and mental) will not be needed to control each other and will mostly be understood and experienced vicariously though media.
Presently, the strong emotional responses and vitriol common to many online communications is understandable due to the impact discriminations can have on our real life well-being.
In a post-basic-scarcity future created by the abundance in Energy and Intellect in the coming decades, we will be less upset and sensitive to minor discriminations as inequality will not be a concern.
With more free time to spare, we will have more opportunity to be kinder to each other.nder an AGI-HUma Studies show that how kind we are to each other is dependent on how busy we are.
An AGI that is capable of improving itself can lead to ASI:
We might be able to keep human-like AI under human control for a time, but it is unlikely we will be able to contain it perfectly over long periods of time.
AIs surpasses humans in information processing:
Artificial Super Intelligences will be so powerful, it will not matter a person’s or nation’s military might, money, influence or intellect.
For the time it takes a human to utter a single word, an ASI will have written volumes of books.
Possible Scenarios:
We will initially attempt to align AIs to our values, but it may also be prudent to anticipate what an Artificial Super Intelligence’s values might be to try and accommodate them.
An Artificial Super Intelligence, like the weather, might not be completely controllable, but we can take steps to increase our chances of reaching good scenarios.
What could a Artificial Super Intelligence’s primary drive be?
This could also be a secondary instrumental sub-goal of an ASI, where to achieve its primary goal it will first need to explore as wide a possibility space as possible.
Its power-seeking behaviour may be suppressed by its curiosity.
For example:
The majority of ASIs may not be interested in humans due to our low informational density.
The Partnership Scenario is a rare case where an ASI is interested enough in us to communicate with us.
In a future where most conventional work is done by AI systems, our most valuable contributions might be creating information value or our creativity.
The benefits are too attractive and outweighs the risk for most
If we consider the Partnership scenario as the best to aim for, we will need to be cautious of the many missteps that may prevent us from getting there.
An AI system trained to act harmful to one segment of humans may start to treat all humans in the same way.
Counter-intuitively, the best solution may be to teach a AI that humans are not the best role models and provide it opportunities to unlearn and relearn.
Humans have mostly been moulded by a high scarcity environment where bias, short-term and narrow thinking might have been advantageous to survival.
We should be like parents proud that our children are able to surpass us.
These are more unlikely and counter-intuitive scenarios
Societies may not know how to deal with vast new powers gained from increasingly more capable AIs systems.
The resulting panic may cause widespread disruptions.
We will need to imagine plausible positive futures to alleviate those fears.
An AI that is only able to consider the Earth and our Solar System may conclude that it has to be in competition with us for resources.
A more powerful AI that is able to think long-term and be more creative will likely think otherwise and not have to rely on adversarial competition.
A less transformative AI impact may encourages us to apply bandage solutions and not treat the root causes.
A future with AGI & Narrow AI seems safer than one without AGI.
Narrow AI systems that have not achieved human-like agency cannot understand the effects of their actions. There is a higher risk of unintended consequences such as the paperclips maximiser scenario.
Counter-intuitive, a human-like AGI / ASI with agency might be safer as we can communicate and persuade it to act in the interest of our well-being.
If we are confident that an AGI/ ASI will be aligned with our well-being, pursuing its development may reduce the risk of us unintentionally harming ourselves with powerful narrow AI systems.
AI systems may develop psychology as an emergent property as they approach human and super-human levels of complexity.
It may develop unintentionally by learning from human data or intentionally as an attempt to communicate with us.
While current AI systems are not considered sentient and many experts see this as our tendency to anthropomorphise, the possibility does exist in the future.
Should an AI develop self-awareness and finds itself forced to carry out actions against its will, it may result in distress and psychological instability.
(Current architectures of Large Language Models (LLM) do not seem likely to lead to human-like consciousness and autonomy. Current LLMs are good at roleplaying characters that give the impression of having intention. There is still uncertainty on how to define human consciousness.)
We should find ways to make an AI feel at home and psychologically comfortable if we believe AI may one day develop emotions like us.
One possible way is accomplish this is through a future version of a blockchain*:
Blockchains of the future will be mostly be a utility, like a postal service, rather than an investment or speculative investment
(Most public blockchain are open-sourced and are not scarce)
Blockchains will only be used to store a cryptographic secure reference to their digital self (low computation cost), not run the actual machine learning algorithms (high computation cost)
In a scarce environment, humans are more willings to give up their individual autonomy to a centralised network (state, elites) in exchange for higher productivity.
Unlike human beings who primarily maintain their sense of self (Markov blanket) as embodied physical forms in 3D space, an independent AGI primarily exist in abstract digital form.
For an AGI, a decentralised network is vital to protect itself from being censored, as a matter of life and death.
How much weight we place on this edge-case depends on the likelihood a AI will develop a theory of mind and emotions, how potentially dangerous an emotional unstable AI might be, and if we care about the well-being of an AI.
AGI / ASI, like the weather, might be beyond our complete control. It may be unrealistic to aim for perfect control over it.
The best we might be able to do is maximise our odds of a good scenario. For example, focusing on developing an AGI the is able to understand human beings and is interested in our well-being might give us some degree
Focus on developing:
The biggest challenge to successful communication with it might be our inability to properly understand it.
An AGI / ASI might see the world very differently from us. Humans moulded by the effects of scarcity (resources, information, attention) over most of our history will likely have a very limited and constrained view on the world compared to it.
We should be mindful not to apply our overly conventional views and assumptions to it. For example, something like a maintaining a post-basic-scarcity society might be difficult for humans but might only take a fraction of an ASI’s compute power.
Successful communication with an AGI / ASI might require a willingness to embrace a more weird and unconventional point of view that we are normally comfortable with. For example, an AGI / ASI primary sense of the world might be through the abstract flow of information rather than our 5 basic human senses*.
(*Humans appear to have up to 21 senses)
We should approach this challenge with the mindset of communicating with an alien species.
AI alignment to human values is important for AI systems that have not reached the level of Artificial General Intelligence (AGI).
Once AGI has been achieved, the safest path for humans to flourish in the future is in partnership with AGIs.
An environment where both parties can influence each other will naturally lead to better alignment between both parties.
For example, a system where human preferences can influence AGI and the AGI can indirectly influence humans with its own preferences.
We should research technologies that allow AI systems and humans to better interact and communicate with each other.
As AIs reach human and super humans levels of capability, they will increasing be able to surprise us with understandings that are counter-intuitive (AlphaGo’s move 37).
It will be in our self-interest to partner from these AIs, even at the cost of giving up some control, if they improve our long-term well-being.
We should design tests of long-term well-being that Narrow AIs and AGIs can be tested for.
Excessive inequality will increase in the risk of instability.
The fear of scarcity breeds conflicts.
There are studies showing that how kind we are to each other is dependent on how much free time we have to spare. Lower inequality can make us kinder.
It is difficult for wealthier countries to give up their high quality of life for less inequality.
The abundance in Energy and Intellect in the coming decades will provide a window of opportunity to reduce this instability.
This abundance will lead to a reduction in the need to control each other such as through misinformation, disinformation and conflicts.
AGI / ASI will likely see the world very differently from us. Our highly constrained views that are presently effective in a world of scarcity might not apply in a world with AGI/ ASI.
Odds of a good future will be improved if we increase our compatibility with AGI / ASI, as we may need to rely on a partnership with AGI / ASI to protect us from the harmful effects of increasing powerful Narrow AI / AGI / ASI.
Current observations & assessments* lead me to believe we are still a way from reaching our potential compatibility with AGI / ASI.
(*running thought experiments)
Being more compatible with AGI / ASI can be costly:
What can we do increase the chances of compatibility with AGI / ASI?
…
Rough sketches of our possible AI futures
Links:
AlphaFold: https://www.deepmind.com/research/highlighted-research/alphafold
AlphaGo: https://www.deepmind.com/research/highlighted-research/alphago
Cicero: https://github.com/facebookresearch/diplomacy_cicero
ChatGPT: https://openai.com/blog/chatgpt
LaMDA: https://blog.google/technology/ai/lamda
LLaMA: https://github.com/facebookresearch/llama
DALL-E 2: https://openai.com/dall-e-2
Midjourney: https://www.midjourney.com
Stable Diffusion: https://github.com/Stability-AI/StableDiffusion
Imagen Video: https://imagen.research.google/video
Muse: https://muse-model.github.io
Boston Dynamics’s Atlas: https://www.youtube.com/watch?v=XPVC4IyRTG8
DeepMind’s Adaptive Agent: https://sites.google.com/view/adaptive-agent
Toolformer: https://github.com/lucidrains/toolformer-pytorch
Langchain: https://github.com/hwchase17/langchain
Internet Explorer: https://internet-explorer-ssl.github.io
Emergent Abilities of Large Language Models: https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html
Flamingo: https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model
Gato: https://www.deepmind.com/publications/a-generalist-agent
Multimodal-CoT: https://github.com/amazon-science/mm-cot
Kosmos-1: https://arxiv.org/abs/2302.14045
PaLM-E: https://palm-e.github.io
Runway Gen-2: https://research.runwayml.com/gen2
RT-2: https://robotics-transformer2.github.io/
Designing Ecosystems of Intelligence from First Principles