| View previous topic :: View next topic |
| Author |
Message |
Allan C Cybulskie
Guest
|
 Posted: Wed Oct 29, 2008 11:53 am Post subject: Re: The wirehead problem |
 |
|
On Oct 23, 8:31 am, Tim Tyler <seemy...@googlemail.com> wrote:
| Quote: | Allan C Cybulskie wrote:
On Oct 22, 12:57 pm, Tim Tyler <seemy...@googlemail.com> wrote:
Allan C Cybulskie wrote:
On Oct 22, 10:00 am, Tim Tyler <seemy...@googlemail.com> wrote:
Allan C Cybulskie wrote:
On Oct 17, 4:20 pm, Tim Tyler <seemy...@googlemail.com> wrote:
Allan C Cybulskie wrote:
To succeed at that is to fail to produce an AI with human level
intelligence, since human level intelligence implies the ability to re-
prioritize and re-interpret goals, and any system that can do that can
do so in such a way that any such prevention will not work.
Where are you getting that from? Intelligence is problem-solving
ability. It doesn't have anything to do with reformulating your
own goals
[...]
there's no reason to think that "ultimate goals" --
whatever they are to you -- won't be at least able to be re-
prioritized.
Yes there is - ultimate goals are, well, ultimate. They
can't be reprioritised, because they are always the only
priority. There is no motivation to change them, since
that would interfere with your existing ultimate goals.
But if you want to solve problems and have human level intelligence,
you'll have to be able to suspend or reprioritize them, or you could
never solve a problem by, say, ignoring pain to complete a task, if
pain is an ultimate goal.
Right - stating the faulty premise up front certainly helps with
the refutation - pain-avoidance is not normally an ultimate goal.
|
As Curt corrected you on, it seems to be for Curt, and he was the one
I was replying to.
Basically, the issue here is NOT over the definition of intelligence
as you assert later, but over what it means for something to be an
ultimate goal. Clearly, we aren't looking at an ultimate goal in the
same way, as I will show next. But, at any rate, dismissing the heart
of the discussion with a "that's not an ultimate goal" should surely
strike you as somehow missing something, no?
And, again, my analysis used YOUR definition of intelligence, but
clearly not your definition of "ultimate goal".
| Quote: | You said later that you think the issue is that I'm using a different
definition of intelligence than you are. That may be true -- I am
sticking, however, to the definition that talks specifically about
what humans do and how they are intelligent, which includes
reprioritizing goals -- but you said it in a section where I was
showing that by YOUR definition of intelligence -- problem-solving --
ultimate goals might have to be reprioritized. So the issue here is
that I'm not certain what sorts of goals you are considering
"ultimate", and you have not reacted either positively or negatively
to my specific examples -- pain, survival, etc -- to show me what you
think of as ultimate goals. So I need you to tell me what sorts of
ultimate goals you are proposing [...]
Ultimate goals can be whatever the programmer chooses to specify.
Big Blue's ultimate goal was winning games of chess. The ultimate
goal of biological organisms is - roughly speaking - leaving
grand-children. You could specify an ultimate goal of depositing
a ton of gold atoms in your back yard. And so on.
|
Ah, see, you are using "ultimate goal" to basically mean "purpose".
That's not how I'm using it. I differentiated proximate goals from
ultimate goals with the intent that ultimate goals were set, hard-
coded goals that didn't necessarily lead to the completion of any task
but are universal things that the robot has to always keep in mind.
By that, pain-avoidance is CLEARLY an ultimate goal in my definition,
because if it can feel pain we can assume that that is a damage
indicator, and the robot should always try to avoid taking damage.
But it has to be able to suspend that if the damage is minor and the
cost to its current task for not suspending it is sufficiently high.
So that's what I meant by ultimate goal.
Now, let's look at considering it as "purpose", and there's a big
problem here: there is no reason that the "purpose" needs to be
necessarily encoded as an explicit goal -- and therefore, something
that can be manipulated -- in the system at all. And therefore, no
reason for what you call the "ultimate goal" to be a goal to the
system at all. For example, Big Blue does not need to know that its
purpose is to win at chess; you can encode the input in any way you
want, so that the system is actually solving a completely different
task than playing chess that gets translated from the output to chess
moves. If it's the case that humans have a purpose of producing
grandchildren -- why not children or descendents instead of
grandchildren? I don't see why that's important -- it's also clear
that we have no clue about that purpose as any necessarily explicit
goal. So using "purpose" to mean "ultimate goal" fails to address
anything relevant; no intelligent agent can operate on a goal that it
does not explicitly have, but my claim was that all intelligent agents
need to be able to operate on goals.
And since you've accepted -- it seems -- that intelligent agents need
to be able to operate on proximate goals, you don't seem to have in
any way refuted the claim that you claim to disagree with.
This, to me, seems to be where the confusion is. Additionally, you
talk about ultimate goal inconsistently, which I'll show in the rest
of the post.
| Quote: |
In short, if you make the AI intelligent, you won't have that level of
control over them.
It doesn't follow. Control by humans over the machines will be via
"ultimate" goals - which won't change. Proximate goals can change -
provided they remain in service of the ultimate goals of the system.
The ultimate goals can provide pretty much whatever level of control
that you can formally describe.
Now, you say above that intelligence is about problem-solving. So
what if in order to solve a problem that you set the robot to, it has
to at least temporarily suspend its ultimate goals to achieve a
proximate goal, something akin to the idea of having to undergo pain
to complete a task?
Ultimate goals are never suspended. The entire point of the
proximate goals is to help attain the ultimate goals.
So if avoiding pain is an ultimate goal, and the robot needed to
experience some pain to complete the task and solve the problem, would
the machine be unable to do so?
That depends on whether the utility function specifies prediction of
future states and temporal discounting. If it has no concept of
future gain at a present cost - as you seem to suggest - it would
be unable to do so.
|
Both you and Curt make the same mistake; I am not suggesting that the
robot cannot calculate future values of pain. I am suggesting that
the future outcomes do not include pain, and if a robot cannot
reprioritize feeling pain to complete a task -- recall my view of
ultimate goal -- then it could not take that action if the action
would cause pain but what would happen if they don't take that action
does not. Thus, an intelligent robot has to be able to accept some
pain or else we have cases where it fails its task because it is
trying to avoid pain. So you can't hardcode pain avoidance as a top
priority that cannot be changed, or else the robot will NEVER be as
smart as a human.
| Quote: |
Then it isn't intelligent; in
principle, it has to be able to accept a small amount of pain to
achieve the goal [...]
But you *just* said the *ultimate* goal was "avoiding pain"! You can't
just have the concept of "goal" refer to one thing in one sentence and
something totally different in the next.
|
So, you are aware that when I use the word "goal" I refer to proximate
as well as ultimate goals -- since I've corrected you on that at least
twice -- and somehow can't manage to do the math from the example that
I was referring to the task that the robot was assigned to do (not
necessarily purpose)?
| Quote: |
If you build a machine whose ultimate goal is finding prime numbers,
and then you tell it to make you a cup of coffee, and it ignores
you, you don't get to claim that the machine is not working properly.
If you /want/ an obedient robot, why not give it the ultimate goal
of obeying you - instead of giving it a different goal - and then
complaining when it doesn't do what you tell it to.
|
I don't see why I can't ask it to obey me and still ask it to only do
specific tasks ... which is not possible by your definition of
ultimate goal. But in neither case was I giving the robot a purpose,
and that's the heart of your objection here.
| Quote: |
There's a difference between "can" and "refuse to" -- so someone who
refuses to accept a little pain to, say, help someone else out -- but
if they simply CANNOT suspend their ultimate goals they aren't human
level intelligent, no matter how useful they may be in limited
domains.
Again, it is not clear where you are getting this from.
Do you have a different definitiion of intelligence from
me - one that mentions reformulating goals?
I'm referring strictly to human level intelligence here. Can humans
do that? Then it's a requirement, full stop.
Humans also use thier intelligence to commit suicide. Must machines
emulate that behaviour to exhibit human-level intelligence?
|
Do robots have to be capable of deciding that to achieve their task it
is better for them to terminate themselves to be intelligent?
Absolutely.
Let's take your colonizing robot. At the end of the task, more metal
is required to finish the task. The robot will not be required after
that task is completed, since it will have completed the only task it
was built to do. It's made of metal and the metal in its body is
sufficient to complete the task. Do you not think that the
intelligent thing to do would be to sacrifice itself to get the metal
to complete the task, knowing that once the metal is available its
last order will complete the task?
Robots are likely to be MORE suicidal than humans, because they'll
likely have a less strong sense of self-preservation than we do.
| Quote: |
What about all the human biases?
http://en.wikipedia.org/wiki/List_of_cognitive_biases
Must machines emulate those too?
|
This gets back to the thing I said at the beginning: the difference
between the Engineering and Cognitive Science view of AI.
For Engineering, we'd like to avoid these, because we want the task
done with the least errors and the most efficiently as possible.
For Cognitive Science, we want a computer model of human cognition and
how that works. So yes, it should have all the errors or else it
isn't properly modelling how humans do it.
That being said, things aren't that simple. Some of those biases are
likely REQUIRED to do what humans do: be a domain-general intelligence
in a less-than deterministic world where we do not know and cannot
know all the information required to make a decision. Part of both
projects is figuring out which "errors" are there because the right
type of intelligence requires that sort of reasoning to make any sense
out of our world.
| Quote: | Additionally, what happens when ultimate goals conflict?
Impossible, in a utilitarian framework.
As soon as you have more than one goal, it is quite possible.
Utilitarian frameworks hide that by proposing one goal -- maximum
"utility" -- but then introduce a lot of interpretation to see what
that utility is. From this, a robot may well decide that it is better
for it and everyone if all humans were killed. See the movie version
of "I, Robot" for an example logic that might lead to that conclusion.
The point was, that you can't possibly have conflicting ultimate goals,
if the system's goal is utilitarian.
Whether a goal leads it to kill all humans rather depends on what the
goal is.
|
As I said, ultilitarianism posits one goal -- maximize utility -- but
has to rely on proximate goals to define what the utility for the
robot is.
| Quote: |
"Utility" cannot be used in a discussion of "ultimate goals"; that
theory would insist that there is only one ultimate goal, but that one
is so vague that it cannot "control" anything.
What?!? On what do you base the ridiculous idea that utility is
necessarily vague?
|
Because utility itself is based on other factors -- ie other goals --
that are rather subjective and may interact with each other in
unexpected ways.
In your article it seems clear because it's talking about economics,
and defines payoffs as monetary (and generally risks as well). In the
real world, things are far more complicated; for example, extra money
is less of a benefit for me than it is for other people, and I have a
tendency to be willing to trade money for time. This works if you
calculate utility, but not in the economic sense. This is why it is
vague -- the goal of "maximize utility" -- because you have to define
utility, which means you have to define payoff and risk, and in a
general purpose and domain-general intelligence -- like in humans --
there are too many goals and factors to do that unambiguously.
I think it is fair for me to limit my claim of vague to the actual
intelligence we are striving for, no?
| Quote: |
http://en.wikipedia.org/wiki/Expected_utility
Note this from your own article:
"The expected utility model fails to provide a good description of how
people make choices in many circumstances because it assumes too much;
humans rarely, if ever, have all the information necessary to make a
decision. In many cases there is no real way to foresee consequences
with any certainty.[4]"
A confused wikipedia author. Expectations are inevitably based on what
knowledge agents have available to them. There is no assumption that
agents have all pertinent information.
|
Actually, you are the one confused; the claim is more that we don't
have the knowledge to make decisions based on it, so it seems unlikely
that we could use that effectively to make decisions in the real
world. This suggests that there might be another mechanism than that
that we are actually using which works better in the world we live in.
| Quote: |
In addition, your own article shows that humans do NOT seem to act
according to this principle, and I was referring specifically to
humans in the example that you derided.
Humans are imperfect rational economic agents. So will machines
be imperfect. You can't actually construct a perfectly rational
economic agent - due to problems such as lack of material resources.
|
Yes, yes, but your objection was invalid because my comment was about
what humans ACTUALLY DO, not what they should do, and you said I was
wrong.
| Quote: |
The expected utility theorem applies to humans - to the extent that
they approximate rational economic agents. That extent is not
overwhelmingly large - but you /can/ model humans as having a
single utility function. Biologists regularly do this to all creatures.
|
I don't think it has worked for psychology, and in addition having it
basically work does not mean that that is what we always do.
| Quote: |
If they can't choose one ultimate goal over another, they
won't be intelligent, and if they can only do so on the basis of a
hard-coded priority of ultimate goals -- see Asimov's rules of
robotics -- then they still aren't intelligent because that choice may
IMPEDE solving the problem they are trying to solve.
That seems like bizarre reasoning to me.
Well, take Asimov's rules:
1) A robot will not allow a human to come to harm.
2) A robot will not allow itself to come to harm, except where that
conflicts with 1).
3) A robot will perform the tasks set to it by a human unless that
conflicts with 1) or 2).
Imagine that the robot is carrying a specific component that is
required to finish its task, and it has a choice where it can enter a
burning building -- almost certainly destroying the component, and it
can't remove it -- to save a human or moving on. Since the First Law
DID include "through inaction", the robot would have to enter the
building to save the human, which would destroy the component and fail
the task. How important that task was being irrelevant in this case.
I can come up with many more examples where you can get into an issue
because it would be better to reprioritize the rules, except the robot
CAN'T do it.
Asimov's laws are multiple. Ultimate goals are necessarily singular -
so in the case of your robot, the ultimate goal would include all of
Asimov's laws. Reprioritising the laws is not reordering any ultimate
goals - it's *changing* the ultimate goal.
|
Now, above, you claimed that we use utility ultimate goals can't
clash. Now you are saying that there can be only one ultimate goal.
Since I explicitly stated that if you can have more than one ultimate
goal they can clash and you replied to that with the idea that utility
stops that, is it any wonder that things got confused? |
|
| |
|
 Back to top |
Allan C Cybulskie
Guest
|
| Posted: Wed Oct 29, 2008 11:57 am Post subject: Re: The wirehead problem |
|
|
On Oct 23, 8:56 pm, c...@kcwc.com (Curt Welch) wrote:
| Quote: | Tim Tyler <seemy...@googlemail.com> wrote:
Allan C Cybulskie wrote:
So if avoiding pain is an ultimate goal, and the robot needed to
experience some pain to complete the task and solve the problem, would
the machine be unable to do so?
That depends on whether the utility function specifies prediction of
future states and temporal discounting. If it has no concept of
future gain at a present cost - as you seem to suggest - it would
be unable to do so.
Exactly. Allan doesn't seem to understand that the normal way all
reinforcement learning systems are built include a function to discount
estimated future gains.
We would never consider a system intelligent if it had no ability to
estimate future rewards and instead only optimized current rewards - which
is exactly the point Allan is making here and in the other message I
haven't been able to replied to yet.
|
Except that my comment was not about being unable to calculate future
rewards, but that those didn't include avoiding pain and if you hard-
code the robot to prefer avoiding pain over EVERYTHING ELSE then it
can't do that thing.
If you don't hard-code it, then there is no goal that you can give the
system that will necessarily prevent it from killing humans. |
|
| |
|
| Back to top |
Alpha
Guest
|
| Posted: Wed Oct 29, 2008 2:03 pm Post subject: Re: The wirehead problem |
|
|
On Oct 29, 4:53 am, Allan C Cybulskie <allan_c_cybuls...@yahoo.ca>
wrote:
| Quote: | On Oct 23, 8:31 am, Tim Tyler <seemy...@googlemail.com> wrote:
snip |
| Quote: | A confused wikipedia author. Expectations are inevitably based on what
knowledge agents have available to them. There is no assumption that
agents have all pertinent information.
Actually, you are the one confused;
|
Yes - that seems to be a recurring theme for Tim <sigh>.
<snip>
| Quote: |
The expected utility theorem applies to humans - to the extent that
they approximate rational economic agents. That extent is not
overwhelmingly large - but you /can/ model humans as having a
single utility function. Biologists regularly do this to all creatures.
|
Biologists, Tim, do not *regularly* model creatures as having a single
utility function; that is false. |
|
| |
|
| Back to top |
Alpha
Guest
|
| Posted: Wed Oct 29, 2008 3:40 pm Post subject: Re: The wirehead problem |
|
|
On Oct 29, 7:52 am, Tim Tyler <seemy...@googlemail.com> wrote:
| Quote: | Alpha wrote:
Allan C Cybulskie <allan_c_cybuls...@yahoo.ca> wrote:
you /can/ model humans as having a
single utility function. Biologists regularly do this to all creatures.
Biologists, Tim, do not *regularly* model creatures as having a single
utility function; that is false.
Sure they do. That's the idea of "fitness" in evolutionary biology.
|
Individual organisms are best thought of as adaptational-executers
rather than as fitness-maximizers. (a more appropo and correct
rendering of the claim by John Tooby and Leda Cosmides, in The
Psychological Foundations of Culture. ins the article: Adaptation-
Executers, not Fitness-Maximizers at the Overcoming Bias site, which
has the following quote:
"Individual organisms are best thought of as adaptation-executers
rather than as fitness-maximizers."
Note the difference: Adaptation-executers vs the corrected by me (via
Buller's work) version using the term adaptational instead of
adaptation (a huge difference as Buller points in his Adapting Minds
book).
| Quote: |
Organisms approximate optimisation processes - and fitness is what
they can be considered to be optimising.
|
Nope: "No human being with the deliberate goal of maximizing their
alleles' inclusive genetic fitness, would ever eat a cookie unless
they were starving. But individual organisms are best thought of as
adaptation-executers, not fitness-maximizers." (Overcoming Bias)
Just replace maximizers with optimizers - same notion.
| Quote: |
That's the central explanatory framework in evolutionary biology.
|
You seem to not know how the Modern Synthesis is is changing, per
Reid's work etc. |
|
| |
|
| Back to top |
Tim Tyler
Guest
|
| Posted: Wed Oct 29, 2008 5:47 pm Post subject: Re: The wirehead problem |
|
|
Allan C Cybulskie wrote:
| Quote: | Ah, see, you are using "ultimate goal" to basically mean "purpose".
That's not how I'm using it. [...]
|
My usage is the one described on:
http://timtyler.org/expected_utility_maximisers/
I feel a bit drowned in words here. However, there's a problem,
in that most of them are either confused, based on misunderstandings,
or otherwise are not very interesting to me. I am deterred from
further attempts at debate by aspects of the style of the discussion.
So, if you want to carry on believing that humans won't be able to
control the intelligent machines which they construct, go for it.
--
__________
|im |yler http://timtyler.org/ tim@tt1lock.org Remove lock to reply. |
|
| |
|
| Back to top |
Alpha
Guest
|
| Posted: Wed Oct 29, 2008 6:13 pm Post subject: Re: The wirehead problem |
|
|
On Oct 29, 8:59 am, Tim Tyler <seemy...@googlemail.com> wrote:
| Quote: | Alpha wrote:
On Oct 29, 7:52 am, Tim Tyler <seemy...@googlemail.com> wrote:
Alpha wrote:
Biologists, Tim, do not *regularly* model creatures as having a single
utility function; that is false.
Sure they do. That's the idea of "fitness" in evolutionary biology.
Individual organisms are best thought of as adaptational-executers
rather than as fitness-maximizers. (a more appropo and correct
rendering of the claim by John Tooby and Leda Cosmides, in The
Psychological Foundations of Culture. ins the article: Adaptation-
Executers, not Fitness-Maximizers at the Overcoming Bias site, which
has the following quote:
"Individual organisms are best thought of as adaptation-executers
rather than as fitness-maximizers."
Yawn:
``Re: "Individual organisms are best thought of as
adaptation-executers rather than as fitness-maximizers".
It's a bit like saying deep blue is an instruction executor,
not an expected chess position utility maximizer.
The statement muddles up the "why" and "how" levels of
explanation.
Executing instructions are *how* chess programs go about
maximizing expected chess position utility.
Of course organisms cannot necessarily maximise their
fitnesses - rather they attempt to maximise their *expected*
fitness, just like other expected utility maximisers.
Tooby and Cosmides go on to argue the even more confused
thesis:
|
No kidding! That is why I replaced "adaptations" with "adaptational",
which is what Buller argues for (to make that sentence I excerpted
true!!!) Yoou need to read a bit more carefully!
| Quote: |
"[Goals such as "maximize your fitness" or "have as many
offspring as possible"] are probably impossible to instantiate
in any computational system."''
...
``As far as I can tell, the idea that organisms maximising
their actual fitnesses is a ridiculous straw man erected
by Tooby and Cosmides for nefarious rhetorical purposes
of their own. Nobody ever actually thought that.''
|
No kidding! That is why I replaced "adaptations" with "adaptational",
which is what Buller argues for (to make that sentence I exceprted
true!!!) You need to read a bit more carefully!
| Quote: |
...
``The Tooby and Cosmides dichotomy between Adaptation-Executers and
Fitness-Maximizers that this blog post is about is a mostly a false
one - based on muddling up "how" and "why" levels of explanation.
Maximising their expected fitness is why organisms behave as they
do. Executing adaptations is how they do it. These different types
of explanations are complimentary, and are not mutually-exclusive.''
-http://www.overcomingbias.com/2007/11/adaptation-exec.html
|
None of which argues against what I actually said.
| Quote: | --
__________
|im |yler http://timtyler.org/ t...@tt1lock.org Remove lock to reply. |
|
|
| |
|
| Back to top |
Allan C Cybulskie
Guest
|
| Posted: Wed Oct 29, 2008 6:46 pm Post subject: Re: The wirehead problem |
|
|
On Oct 29, 12:06 pm, Tim Tyler <seemy...@googlemail.com> wrote:
| Quote: | Alpha wrote:
On Oct 29, 7:52 am, Tim Tyler <seemy...@googlemail.com> wrote:
Organisms approximate optimisation processes - and fitness is what
they can be considered to be optimising.
Nope: "No human being with the deliberate goal of maximizing their
alleles' inclusive genetic fitness, would ever eat a cookie unless
they were starving. But individual organisms are best thought of as
adaptation-executers, not fitness-maximizers." (Overcoming Bias)
Just replace maximizers with optimizers - same notion.
The word "expected" it needed. Biologists do not /literally/
think that organisms maximise their own fitnesses.
They think they maximise their *expected* fitnesses -
based on some cognitive model of the consequences of
their actions.
|
I'm actually quite willing to think that biologists do not do that,
since in biology the term "organisms" refers to things that range from
viruses and bacteria through plants and insects and finally to animals
with brains (including humans). Most of the things on that list are
not considered by biology to have the ability to have cognitive models
or even understand the consequences of their actions. |
|
| |
|
| Back to top |
Alpha
Guest
|
| Posted: Wed Oct 29, 2008 7:23 pm Post subject: Re: The wirehead problem |
|
|
On Oct 29, 9:10 am, Tim Tyler <seemy...@googlemail.com> wrote:
| Quote: | Alpha wrote:
You seem to not know how the Modern Synthesis is is changing, per
Reid's work etc.
I believe I offered a brief review of Reid to you once before - see:
http://groups.google.com/group/comp.ai.philosophy/msg/f2129ec47bcce15...
|
Yes - and withut reading the book completely (your having said you
waded through only chapter 1, it is difficult to gt a grasp of what
he is talking about. It is like reading the introduction chapter of a
QM text; don't expect to know much about QM from that!
| Quote: | --
__________
|im |yler http://timtyler.org/ t...@tt1lock.org Remove lock to reply. |
|
|
| |
|
| Back to top |
Tim Tyler
Guest
|
| Posted: Wed Oct 29, 2008 7:52 pm Post subject: Re: The wirehead problem |
|
|
Alpha wrote:
| Quote: | Allan C Cybulskie <allan_c_cybuls...@yahoo.ca> wrote:
you /can/ model humans as having a
single utility function. Biologists regularly do this to all creatures.
Biologists, Tim, do not *regularly* model creatures as having a single
utility function; that is false.
|
Sure they do. That's the idea of "fitness" in evolutionary biology.
Organisms approximate optimisation processes - and fitness is what
they can be considered to be optimising.
That's the central explanatory framework in evolutionary biology.
--
__________
|im |yler http://timtyler.org/ tim@tt1lock.org Remove lock to reply. |
|
| |
|
| Back to top |
Tim Tyler
Guest
|
| Posted: Wed Oct 29, 2008 8:59 pm Post subject: Re: The wirehead problem |
|
|
Alpha wrote:
| Quote: | On Oct 29, 7:52 am, Tim Tyler <seemy...@googlemail.com> wrote:
Alpha wrote:
Biologists, Tim, do not *regularly* model creatures as having a single
utility function; that is false.
Sure they do. That's the idea of "fitness" in evolutionary biology.
Individual organisms are best thought of as adaptational-executers
rather than as fitness-maximizers. (a more appropo and correct
rendering of the claim by John Tooby and Leda Cosmides, in The
Psychological Foundations of Culture. ins the article: Adaptation-
Executers, not Fitness-Maximizers at the Overcoming Bias site, which
has the following quote:
"Individual organisms are best thought of as adaptation-executers
rather than as fitness-maximizers."
|
Yawn:
``Re: "Individual organisms are best thought of as
adaptation-executers rather than as fitness-maximizers".
It's a bit like saying deep blue is an instruction executor,
not an expected chess position utility maximizer.
The statement muddles up the "why" and "how" levels of
explanation.
Executing instructions are *how* chess programs go about
maximizing expected chess position utility.
Of course organisms cannot necessarily maximise their
fitnesses - rather they attempt to maximise their *expected*
fitness, just like other expected utility maximisers.
Tooby and Cosmides go on to argue the even more confused
thesis:
"[Goals such as "maximize your fitness" or "have as many
offspring as possible"] are probably impossible to instantiate
in any computational system."''
....
``As far as I can tell, the idea that organisms maximising
their actual fitnesses is a ridiculous straw man erected
by Tooby and Cosmides for nefarious rhetorical purposes
of their own. Nobody ever actually thought that.''
....
``The Tooby and Cosmides dichotomy between Adaptation-Executers and
Fitness-Maximizers that this blog post is about is a mostly a false
one - based on muddling up "how" and "why" levels of explanation.
Maximising their expected fitness is why organisms behave as they
do. Executing adaptations is how they do it. These different types
of explanations are complimentary, and are not mutually-exclusive.''
- http://www.overcomingbias.com/2007/11/adaptation-exec.html
--
__________
|im |yler http://timtyler.org/ tim@tt1lock.org Remove lock to reply. |
|
| |
|
| Back to top |
Tim Tyler
Guest
|
| Posted: Wed Oct 29, 2008 9:06 pm Post subject: Re: The wirehead problem |
|
|
Alpha wrote:
| Quote: | On Oct 29, 7:52 am, Tim Tyler <seemy...@googlemail.com> wrote:
Organisms approximate optimisation processes - and fitness is what
they can be considered to be optimising.
Nope: "No human being with the deliberate goal of maximizing their
alleles' inclusive genetic fitness, would ever eat a cookie unless
they were starving. But individual organisms are best thought of as
adaptation-executers, not fitness-maximizers." (Overcoming Bias)
Just replace maximizers with optimizers - same notion.
|
The word "expected" it needed. Biologists do not /literally/
think that organisms maximise their own fitnesses.
They think they maximise their *expected* fitnesses -
based on some cognitive model of the consequences of
their actions.
Organisms may have inaccurate expectations - such as, in
this case, the expectation that a famine is about to come -
and so they should hang on to every last calorie.
A cookie eating organism may well be maximising their
expected fitness - but their expectations about the
consequences of their actions might be wrong.
--
__________
|im |yler http://timtyler.org/ tim@tt1lock.org Remove lock to reply. |
|
| |
|
| Back to top |
Tim Tyler
Guest
|
| Posted: Wed Oct 29, 2008 9:10 pm Post subject: Re: The wirehead problem |
|
|
Alpha wrote:
| Quote: | You seem to not know how the Modern Synthesis is is changing, per
Reid's work etc.
|
I believe I offered a brief review of Reid to you once before - see:
http://groups.google.com/group/comp.ai.philosophy/msg/f2129ec47bcce152?hl=en
--
__________
|im |yler http://timtyler.org/ tim@tt1lock.org Remove lock to reply. |
|
| |
|
| Back to top |
Tim Tyler
Guest
|
| Posted: Thu Oct 30, 2008 12:00 am Post subject: Re: The wirehead problem |
|
|
Allan C Cybulskie wrote:
| Quote: | On Oct 29, 12:06 pm, Tim Tyler <seemy...@googlemail.com> wrote:
The word "expected" it needed. Biologists do not /literally/
think that organisms maximise their own fitnesses.
They think they maximise their *expected* fitnesses -
based on some cognitive model of the consequences of
their actions.
I'm actually quite willing to think that biologists do not do that,
since in biology the term "organisms" refers to things that range from
viruses and bacteria through plants and insects and finally to animals
with brains (including humans). Most of the things on that list are
not considered by biology to have the ability to have cognitive models
or even understand the consequences of their actions.
|
Only entities with cognitive models use them in the way I described.
Organisms with more primitive links between their sensors and
actuators do the best they can with what they have.
--
__________
|im |yler http://timtyler.org/ tim@tt1lock.org Remove lock to reply. |
|
| |
|
| Back to top |
|
