So consider the fact that machines are of a special hardware that can be updated at an insane rate. The point is that they could drive their own “evolution.” So, if the first generation began with the equivalent of a human consciousness, how long would it be until they began to expand upon it (indeed, granted human consciousness would it be able to?) Consciouesness isn’t simply a threshold that one passes. Between species, I argue, there are different levels of consciousness, and perhaps even between individuals. If a dolphin is say 60% “conscious” (arbitrary), I am 99.8% conscious, and my friend is 99.9% conscious, it is easily conceivable that something could develop and harness a consciousness that is far more profound than anything a human being has reached.
Now, this is hard to imagine because, “What in the world would that be like?” This is sort of what the original question was driving at. Now considers us low-conscious individuals trying to imagine the mind of a high-conscious being. It would be comparable to a dolphin trying to anticipate a human’s behavior (or you can imagine your own comparison). Aside from the simple processing advantages that these machines might have (obvious near-instant computations, etc.) what sort of extra sensory perception might it develop? Emotional characteristics? Problem-solving capabilities?
Now, carrying on with the scenario and entering a dangerous zone of abstraction, the generations would progress in a sort of exponential behavior, approaching something called singularity where nothing limits the growth except for the physical boundaries of the universe. Would it ultimately be in the interest of the “Singular” minds to help humans? Would the machines lock themselves into some sort of zen-like state of perpetual euphoria for eternity while we simply stared at them and wondered why they didn’t seem to be working? Essentially, considering this “singular” machine as an extension of the universe, it becomes almost godly. How do god treat us?
More eerily, if these machines are invented, but we progress down this same path – how do we treat ourselves?