Guest
|
 Posted: Fri Nov 14, 2003 10:09 pm Post subject: A Hierarchical Architecture for Software Agents-Part 1 |
 |
|
Abstract:
We describe the design and use of a hierarchically structured autonomous
software agent.
Introduction:
In a series of papers(Kansas Acad. Sci. Transactions, vol. 100, no. 3, pg
85 and vol. 102, no. 3, pg 117) we have described autonomous software agents
(Asa) which can automatically monitor and control a typical scientific
experiment or an industrial plant. Our original program assumed a "flat"
unstructured memory but a hierarchical organization may be needed in order to
limit computational complexity to an acceptable level for practical (large
scale) systems.
The present hierarchical architecture consists of three major elements.
The first one (or more) layers of the hierarchy is an array of classifiers
which accepts inputs at each time step and compares them with stored
categories. The next component in the hierarchy is a multilayered network
of sequence learners/classifiers. As each catagory at each time step is
evaluated and relayed from the lowest layers of the hierarchy the developing
pattern is compared with stored temporal sequences. The best matching
sequences output a set of "value measures." Our architecture bears some
resemblance to Amit>s recent work (Amit and Mataric, Int. Conf. on
Development
and Learning, MIT, June 12, 2002). The topmost module in the hierarchy
combines all of the values input from the sequence classifiers below it and
produces a single scalar estimated utility.
Categorizers:
The lowest layer(s) in our architecture is a set of classifiers which
accepts inputs at each time step, vector IN, and compares them with stored
categories (vectors, INi). In our present implementation case-based
reasoners perform this operation. A vector dot product (or other) similarity
measure is taken between the (normalized) input vector IN, the index, and
each of the stored prototype case vectors, INi. The best match or,
alternatively, the k nearest neighbors, is broadcast to the next layer. If
the
current input vector, IN, is close enough to the best matching cases then
those cases are modified slightly to more closely resemble IN. If the i>th
case, INi, is the average of k previous input vectors then the new input, IN,
will adjust it further according to the formula:
new vector INi = old vector INi + (1/(k+1))*(vector IN - old vector INi)
followed by renormalization of INi. If no match is close enough then a
completely new category is defined by simply recording IN.
A second layer of classifiers accepts the match>s output from the first
layer as its own input vector, and is implemented in the same way as layer 1,
creating a hierarchical set of categories. (Additional layers of categories
can be employed if required.)
For vision data preprocessing might involve modules which shift, rotate,
reflect, and scale each of the recorded category prototypes when comparing
then with input, much in the manor of time warping.
Sequences:
The second component in our agent architecture is a network of sequence
learners. As each catagory at each time step is passed on from the
categorizing subsystem the developing temporal sequence is compared with
stored
sequences (and time shifted/"warped" sequences) in the next higher layer(s)
of the hierarchy. (Perhaps weighting each previous input less and less with
time.) In the present experiment this module is also implemented as a set of
case-based reasoners.
Each input pattern is compared with all stored sequences (cases) and
with
time warped sequences. The nearest matching case (or the k nearest matches)
outputs a degree of match measure (the dot product similarity measure) to the
next layer in the hierarchy.
Sequences of sequences:
Additional layers of sequence learners are stacked one above the other
thus learning a hierarchically structured "sequence of sequences," each
covering a longer time period than the one before. The top most sequence
classifying layer outputs a set of "value measures" to the final module in
the hierarchy. These "values" are things like system "health", damage,
energy supply, foresight, speed, etc. and are detected in the input signal
by hard wired feature detectors. Foresight, for example, is measured based
upon the sequence>s ability to predict the next one or more inputs,
vector IN(t+1), vector IN(t+2), etc. (again, judged by a similarity measure.)
It seems reasonable to assume that learning how the world evolves is a
useful step toward deciding how to act. Prediction of the evolving state of
the world is made easier by the fact that (some) feedback is immediate.
That is, we immediately see what the next state of our sensory input is, next
vector IN, and can compare it with our case predictions, vector INi(t+1).
----- Posted via NewsOne.Net: Free (anonymous) Usenet News via the Web -----
http://newsone.net/ -- Free reading and anonymous posting to 60,000+ groups
NewsOne.Net prohibits users from posting spam. If this or other posts
made through NewsOne.Net violate posting guidelines, email abuse@newsone.net |
|
