The Bellman Award is given for distinguished career contributions to the theory or application of automatic control. It is the highest recognition of professional achievement for US control systems engineers and scientists. The recipient must have spent a significant part of his/her career in the USA. The awardee is strongly encouraged to give a plenary presentation at the ACC Awards Luncheon.

## You are here

# Richard E. Bellman Control Heritage Award

## John S. Baras

**John S. Baras** holds a permanent joint appointment as professor in the department of electrical and computer engineering and the Institute for Systems Research. He was the founding director of ISR, which is one of the first six National Science Foundation engineering research centers. Dr.

Dear President Masada, colleagues, students, ladies and gentlemen.

I am deeply moved by this award and honor, and truly humbled to join a group of such stellar members of our extended systems and control community, several of whom have been my mentors, teachers and role models throughout my career.

I am grateful to those who nominated me and supported my nomination and to the selection committee for their decision to honor my work and accomplishments.

I was fortunate through my entire life to receive the benefits of exceptional education. From special and highly selective elementary school and high school back in Greece, to the National Technical University of Athens for my undergraduate studies and finally to Harvard University for my graduate studies. My sincere and deep appreciation for such an education goes to my parents who distilled in me a rigorous work ethic and the ambition to excel, my teachers in Greece for the sound education and training in basic and fundamental science and engineering and to my teachers and mentors at Harvard and MIT (Roger Brockett, Sanjoy Mitter and the late Jan Willems) and the incredibly stimulating environment in Cambridge in the early 70’s.

Many thanks are also due to my students and colleagues at the University of Maryland, in the US and around the world, and in particular in Sweden and Germany, for their collaboration, constructive criticism and influence through the years. Several are here and I would like to sincerely thank you all very much.

I am grateful to the agencies that supported my research: NSF, ARO, ARL, ONR, NRL, AFOSR, NIST, DARPA, NASA. I am particularly grateful to NSF for the support that helped us establish the Institute for Systems Research (ISR) at the University of Maryland in 1985, and to NASA for the support that helped us establish the Maryland Center for Hybrid Networks (HyNet) in 1992.

I would also like to thank many industry leaders and engineers for their advice, support, and collaboration during the establishment and development of both the ISR and HyNet to the renowned centers of excellence they are today.

Most importantly I am grateful to my wife Mary, my partner, advisor and supporter, for her love and selfless support and sacrifices during my entire career.

When I came to the US in 1970 I was debating whether to pursue a career in Mathematics, Physics or Engineering. The Harvard-MIT exceptional environment allowed me freedom of choice. Thanks to Roger Brockett I was convinced that systems and control, our field, would be the best choice as I could pursue all of the above. It has indeed proven to be a most exciting and satisfying choice. But there were important adjustments that I had to make and lessons I learned.

I did my PhD thesis work on infinite dimensional realization theory, and worked extensively with complex variable methods, Hardy function algebras, the famous Carleson corona theorem and several other rather esoteric math. From my early work at the Naval Research Laboratory in Electronic Warfare (the “cross-eye” system) and in urban traffic control (adaptive control of queues) I learned, the hard way, the difficulty and critical importance of building appropriate models and turning initially amorphous problems to models amenable to systems and control thinking and methods. I learned the importance of judiciously blending data-based and model-based techniques.

In the seventies, I took a successful excursion into detection, estimation and filtering with quantum mechanical models, inspired by deep space laser communication problems, where my mathematical physics training at Harvard came in handy. I then worked on nonlinear filtering, trying to understand how physicists turned nonlinear inference problems to linear ones and investigate why we could not do the same for nonlinear filtering and partially observed stochastic control. This led me to unnormalized conditional densities, the Duncan-Mortensen-Zakai equation and to information states. This led me naturally to construct nonlinear observers as asymptotic limits of nonlinear filtering problems and the complete solution of the nonlinear robust output feedback control problem (nonlinear H-infinity problem) via two coupled Hamilton Jacobi Bellman equations. We even investigated the development of special chips to implement real-time solutions, a topic we are revisiting currently.

With the development and progress of the ISR I worked on many problems including: speech and image compression breaking the Shannon separation of source and channel coding, manufacturing processes, network management, communication network protocols, smart materials (piezoelectric, shape memory alloys), mobile wireless network design, network security and trust, and more recently human-machine perception and cognition, networked control systems, networked cyber-physical systems, combining metric temporal logic and reachability analysis for safety, collaborative decision management in autonomous vehicles and teams of humans and robots, new analytics for learning and for the design of deep learning networks mapping abstractions of the brain cortex, quantum control and computing.

Why I am telling you about all these diverse topics? Not to attract your admiration. But because at the heart of all my works are fundamental principles and methods from systems and controls, often appropriately extended and modified. Even in my highest impact (economic and social) work in conceiving, demonstrating and commercializing Internet over satellite services (with billions of sales world-wide – remember me when you use Internet in planes over oceans), we modified the flow control algorithm (the TCP) and the physical path, to avoid having TCP interpret the satellite physical path delay as congestion. That is we used systems and control principles.

Our science and engineering, systems and control, has some unparalleled unifying power and efficiency. That is, if we are willing to build the new models required by the new applications (especially models requiring a combination of multiple physics and cyber logic) and if we are willing to learn and apply the incredible new capabilities and technologies that are developed in information technology and materials. As is apparent especially in this conference (ACC), and in the CDC conference, by any measure, our field is exceptionally alive and well and continues to surprise many other disciplines by its contributions and accomplishments, which now extend even in biology, medicine and healthcare. So for the many young people here, please continue the excitement, continue getting involved in challenging and high impact problems, and continue the long tradition and record of accomplishments we have established for so many years. And most importantly continue seeking the common ground and unification of our methods and models.

Let me close with what I consider some major challenges and promising broad areas for the next 10 years or so:

1) Considering networked control systems we need to understand what we mean by a “network” and the various abstractions and system aspects involved. Clearly there are more than one dynamic graphs involved. This needs new foundations for control, communication, information, computing.

2) Systems and control scientists and engineers are the best qualified to develop further the modern field of Model-Based Systems Engineering (MBSE): the design, manufacturing/implementation and operation of complex systems with heterogeneous physical, cyber components and even including humans.

3) The need for analog computing is back, for example in real-time and progressive learning and in CPS. Some of the very early successes of control were implemented in analog electromechanical systems due to the need for real-time behavior. Yet we do not have a synthesis theory and methodology for such systems due to the heterogeneous physics that may be involved. Nothing like we have for VLSI.

Thank you all very much! This is indeed a very special day for me!

## Jason L. Speyer

**Charles Stark**)

**Draper**, whose second volume of his three sequence series on Instrument Engineering (1952) was one the first books on what we know as Classical Control covering such topics as

**[1]****Evens**root locus,

**Bode**plots, Nyquist criterion, and

**Nichols**charts. Doc

**Draper**instituted an undergraduate course in classical control that I took my junior year. This inspired me to take a graduate course and write my undergraduate thesis in controls.

**Richard Bellman**for global sufficiency of an optimal trajectory and the Pontryagin Maximum principle inspired by the deficiency of dynamic programing to solve certain classes of optimization problems. The Bushaw problem of determining the minimum time to the origin of a double integrator was just such a problem, since the optimal return function in dynamic programing is

*not*differentiable at the switching curve and the Bellman theory did not apply. Interestingly, for my bachelor’s thesis I applied the results of the Bushaw problem to the minimum time problem of bringing the yaw and yaw rate of an aircraft to the origin. However, at that time I had no idea about the ramification of the Bushaw problem to optimization theory. I also learned of the work of

**Rudolf**

**Kalman**in estimation, the work of

**Arthur Bryson**and Henry Kelley in the development of numerical methods for determining optimal constrained trajectories, and J. Halcombe (Hal) Laning and Richard Battin on the determination of orbits for moon rendezvous.

**Art Bryson**was a consultant. There, I worked with a student of Bryson, Walter Denham. We were contracted by MIT’s Instrumentation Laboratory, monitored by Richard Battin, to enhance the Apollo autonomous navigation system over the trans-Lunar orbit. We developed a scheme for determining the optimal angle-measurement sequence between the best stars in a catalogue and near and far horizons of the Earth or the Moon using a sextant. This angle-measurement sequence minimized some linear function of the terminal value of the error covariance of position and velocity near the Earth or Moon. Our optimization scheme, which required a matrix dynamic constraint, seemed to be a first. This scheme, used in the Apollo autonomous navigation system, was tested on Apollo 8, and used on every mission thereon. My next task at Raytheon was working on neighboring optimal guidance scheme. This work was with

**Art Bryson**and

**John Breakwell**. I remember travelling to Lockheed’s Palo Alto Research Laboratory and meeting with John, the beginning of a long and delightful collegial relationship.

**Art Bryson**to take me on as a graduate student at Harvard, supported by the Raytheon Fellowship program. To understand the intellectual level I had to contend with, on my doctorial preliminary exam committee, three of the four examiners were recipients of the Richard E. Bellman Control Heritage Award;

**Art Bryson, Larry (Yu-Chi) Ho,**and

**Bob (Kumpati) Narendra,**all of whom have been my life time colleagues. I was also fortunate to take a course taught by

**Rudy Kalman**. Surprisingly, he taught many of the controls areas he had pioneered, except filtering for Gauss-Markov systems (the Kalman filter); the Aizerman conjecture, the Popov criterion and Lyopunov functions, duality in linear systems, optimality for linear-quadratic systems, etc. After finishing my PhD thesis on optimal control problems with state variable inequality constraints, I returned to Raytheon. Fortunately,

**Art Bryson**made me aware of some interest at Raytheon in using modern control theory for developing guidance laws for a new missile. At Raytheon’s Missile Division I worked with Bill O’Halloran on the homing missile guidance system where Bill worked on development of the Kalman filter and I worked on the development of the linear-quadratic

*closed-form*guidance gains that had to include the nonminimal phase autopilot. This homing missile, the Patriot missile system, appears to be the first fielded system using modern control theory.

[1] Boldface names are recipients of the Richard E. Bellman Control Heritage Award.

## Thomas F. Edgar

## Dimitri P. Bertsekas

Dimitri P. Bertsekas' undergraduate studies were in engineering at the National Technical University of Athens, Greece. He obtained his MS in electrical engineering at the George Washington University, Wash. DC in 1969, and his Ph.D. in system science in 1971 at the Massachusetts Institute of Technology.

## A. Stephen Morse

A. Stephen Morse was born in Mt. Vernon, New York. He received a BSEE degree from Cornell University, MS degree from the University of Arizona, and a Ph.D. degree from Purdue University. From 1967 to 1970 he was associated with the Office of Control Theory and Application (OCTA) at the NASA Electronics Research Center in Cambridge, Mass. Since 1970 he has been with Yale University where he is presently the Dudley Professor of Engineering.

## Arthur J. Krener

Arthur J. Krener received the PhD in Mathematics from the University of California,

Berkeley in 1971. From 1971 to 2006 he was at the University of California, Davis. He

retired in 2006 as a Distinguished Professor of Mathematics. Currently he is a Distinguished Visiting Professor in the Department of Applied Mathematics at the Naval Postgraduate School.

His research interests are in developing methods for the control and estimation of nonlinear dynamical systems and stochastic processes.

It is a honor to receive the 2012 Richard E. Bellman Control Heritage Award. I am deeply humbled to join the very distinguished group of prior winners. At this conference there are so many people whose work I have admired for years. To be singled out among this

group is a great honor.

I did not know Richard Bellman personally but we are all his intellectual descendants. Years ago my first thesis problem came from Bellman and currently I am working on numerical solutions to Hamilton-Jacobi-Bellman partial differential equations.

I began graduate school in mathematics at Berkeley in 1964, the year of the Free Speech Movement. After passing my oral exams in 1966, I started my thesis work with R. Sherman Lehman who had been a postdoc with Bellman at the Rand Corporation in the 1950s. Bellman and Lehman had worked on continuous linear programs also called bottleneck problems in Bellman’s book on Dynamic Programming. These problems are dynamic versions of linear

programs, with linear integral transformations replacing finite dimensional linear transformations. At each frozen time they reduce to a standard linear program. Bellman and Lehman had worked out several examples and found that often the optimal solution was basic, at each time an extreme point of the set of feasible solutions to the time frozen linear program. These extreme points moved with time and the optimal solution would stay on one moving extreme point for awhile and then jump to another. It would jump from one bottleneck to another.

Lehman asked me to study this problem and find conditions for this to happen. We thought that it was a problem in functional analysis and so I started taking advanced courses in this area. Unfortunately about a year later Lehman had a very serious auto accident and lost the ability to think mathematically for some time. I drifted, one of hundreds of graduate students in Mathematics at that time. Moreover, Berkeley in the late 1960s was full of distractions and I was distractable. After a year or so Lehman recovered and we started to meet regularly. But then he had a serious stroke, perhaps as a consequence of the accident, and I was on my own again.

I was starting to doubt that my thesis problem was rooted in functional analysis. Fortunately I had taken a course in differential geometry from S. S. Chern, one of the pre-eminent geometers of his generation. Among other things, Chern had taught me about the Lie bracket. And one of my graduate student colleagues told me that I was trying to prove a bang-bang theorem in Control Theory, a field that I had never heard of before. I then realized that my problem was local in nature and intimately connected with flows of vector fields so the Lie bracket was an essential tool. I went to Chern and asked him some questions about the range of flows of multiple vector fields. He referred me to Bob Hermann who was visiting the Berkeley Physics Department at that time.

I went to see Hermann in his cigar smoked-filled office accompanied by my faithful companion, a German Shepherd named Hogan. If this sounds strange, remember this was Berkeley in the 1960s. Bob was welcoming and gracious, he gave me galley proofs of his forthcoming book which contained Chow’s theorem. It was almost the theorem that I had been groping for. Heartened by this encounter I continued to compute Lie brackets in the hope of proving a bang-bang theorem.

Time drifted by and I needed to get out of graduate school so I approached the only math faculty member who knew anything about control, Stephen Diliberto. He agreed to take me on as a thesis student. He said that we should meet for an hour each week and I should tell him what I had done. After a couple of months, I asked him what more I needed to do to get a PhD. His answer was ”write it up”. My ”proofs” fell apart several times trying to accomplish this. But finally I came up with a lemma that might be called Chow’s theorem with drift that allowed me to finish my thesis.

I am deeply indebted to Diliberto for getting me out of graduate school. He also did another wonderful thing for me, he wrote over a hundred letters to help me find a job. The job market in 1971 was not as terrible as it is today but it was bad. One of these letters landed on the desk of a young full professor at Harvard, Roger Brockett. He had also realized that the Lie bracket had a lot to contribute to control. Over the ensuing years, Roger has been a great supporter of my work and I am deeply indebted to him.

Another Diliberto letter got me a position at Davis where I prospered as an Assistant Professor. Tenure came easily as I had learned to do independent research in graduate school. I brought my dog, Hogan, to class every day, he worked the crowds of students and boosted my teaching evaluations by at least a point. After 35 wonderful years at Davis, I retired and joined the Naval Postgraduate School where I continue to teach and do research. I am indebted to these institutions and also to the NSF and the AFOSR for supporting my career.

I feel very fortunate to have discovered control theory both for the intellectual beauty of the subject and the numerous wonderful people that I have met in this field. I mentioned a few names, let me also acknowledge my intellectual debt to and friendship with Hector Sussman, Petar Kokotovic, Alberto Isidori, Chris Byrnes, Steve Morse, Anders Lindquist, Wei Kang and numerous others.

In my old age I have come back to the legacy of Bellman. Two National Research Council Postdocs, Cesar Aguilar and Thomas Hunt, have been working with me on developing patchy methods for solving the Hamilton-Jacobi-Bellman equations of optimal control. We haven’t whipped the ”curse of dimensionality” yet but we are making it nervous.

The first figure shows the patchy solution of the HJB equation to invert a pendulum. There are about 1800 patches on 34 levels and calculation took about 13 seconds on a laptop. The algorithm is adaptive, it adds patches or rings of patches when the residual of the HJB equation is too large. The optimal cost is periodic in the angle. The second figure shows this. Notice that there is a negatively slanted line of focal points. At these points there is an optimal clockwise and an optimal counterclockwise torque. If the angular velocity is large enough then the optimal trajectory will pass through the up position several times before coming to rest there.

What are the secrets to success? Almost everybody at this conference has deep mathematical skills. In the parlance of the NBA playoffs which has just ended, what separates researchers is “shot selection” and ”follow through”. Choosing the right problem at the right time and perseverance, nailing the problem, are needed along with good luck and, to paraphrase the Beatles, ”a little help from your friends”.

## Manfred Morari

Manfred Morari was appointed head of the Department of Information Technology and Electrical Engineering at ETH Zurich in 2009. He was head of the Automatic Control Laboratory from 1994 to 2008. Before that he was the McCollum-Corcoran Professor of Chemical Engineering and Executive Officer for Control and Dynamical Systems at the California Institute of Technology. He obtained the diploma from ETH Zurich and the Ph.D. from the University of Minnesota, both in chemical engineering. His interests are in hybrid systems and the control of biomedical systems.

Usually when you are nominated for an award you know about it or – at least – you have a suspicion – for example, when somebody asks you for your CV, but you are sure that they are not interested in hiring you. This award came to me as a total surprise. Indeed I had written a letter of support for another most worthy candidate. So, when I received Tamer Başar’s email I thought that it was to inform me that this colleague had won. Who was actually responsible for my nomination? Several of my former graduate students! So, not only were they responsible for doing the work that qualified me for the award, they were even responsible for my getting it!

Over the course of my career I was fortunate to have worked with a fantastic group of people and I am very proud of them: 64 Phd Students to date and about 25 postdocs. 27 of them are holding professorships all over the world – from the Korean Advanced Institute of Science and Technology KAIST in the East to Berkeley and Santa Barbara in the West from the Norwegian Technical University and the U Toronto in the North to the Technion in Israel and the Instituto Tecnologico de Buenos Aires in the South. Many others are now in industry, about 15 in Finance, Management Consulting and Legal, holding positions of major responsibility. I regard this group of former co-workers as my most important legacy.

This award means a lot to me because of the awe-inspiring people who received it in the past. I remember Hendrik Bode receiving the inaugural award in 1979. I remember Rutherford Aris, one of my PhD advisors at the University of Minnesota receiving it in 1992. Aris had actually worked and published with Richard Bellman. I remember Harmon Ray receiving it in 2000, my colleague and mentor at the University of Wisconsin.

Receiving this award made me also reflect on what I felt our major contributions were in these 34 years since I started my career as an Asst. Prof at Wisconsin. In what way was our work important? I was reminded of a dinner conversation a few months back with a group of my former PhD students who had joined McKinsey after graduating from ETH. One of them told me that our group had supplied more young consultants to McKinsey Switzerland than any other institute of any university in Switzerland. He also talked informally about the results of a survey done internally on what may be the main traits characterizing a CEO. It is not charm. It is not tactfulness and sensitivity. It is not intelligence. The only common trait seems to be that in their past these CEOs headed a division that experienced unusual growth. For example, the CEO of a telecom company had headed the mobile phone division. All the CEOs seemed to have been at the right place at the right time.

Similar considerations may apply to doing research and to heading a research group. Richard Hamming, best known for the Hamming code and the Hamming window, wrote in a wonderful essay: “If you are to do important work then you must work on the right problem at the right time and in the right way. Without any one of the three, you may do good work but you will almost certainly miss real greatness….”

So, what are the right problems? Eric Sevareid, the famous CBS journalist once quipped: “The chief cause of problems is solutions.” We were never interested in working on problems solely for their mathematical beauty. We always wanted to solve real practical problems with potential impact. Several times we were lucky to be standing at a turning point, ready to embark on a new line of research before the community at large had recognized it. Let me share with you three examples.

Around 1975, when I started my PhD at the University of Minnesota, interest in process control was just about at an all-time low. In 1979 this conference, which was then called the Joint Automatic Control Conference, had barely 300 attendees. The benefits of optimal control and the state space approach had been hyped so much for more than a decade that disillusionment was unavoidable. Many people advised me not to commence a thesis in process control. But my advisor George Stephanopoulos convinced me that the reason for all the disappointment was that people had been working on the wrong problem. The problem was not how to design controllers for poorly designed system but how to design systems such that they are easy to control. The work that was started at that time by us and several other groups provided valuable insights that are in common use today and set off a whole research movement with special sessions, special journal issues and even separate workshops and conferences.

The second example is our work on Internal Model Control (IMC) and Robust Control. In the early 1980s the term “robust control” did not exist or, at least, it was not widely used and accepted. From our application work and influenced by several senior members of our community we had become convinced that model uncertainty is a critical obstacle affecting controller design. We discovered singular values and the condition number as important indicators before we learned that these were established mathematical quantities with established names. In 1982 at a workshop in Interlaken I met John Doyle, Gunter Stein and essentially everybody else who started to push the robust control agenda. Indeed it was there that Jürgen Ackerman made the researchers in the West aware of the results of Kharitonov. A year later I went to Caltech, John Doyle followed soon afterwards and an exciting research collaboration commenced that lasted for almost a decade. We also cofounded the Control and Dynamical Systems option/department at that time.

The third example is our more recent work on Model Predictive Control (MPC) and Hybrid Systems. As I returned to Switzerland 17 years ago, I moved from a chemical to an electrical engineering department. I was thrown into a new world of systems with time constants of micro- or even nanoseconds rather than the minutes or hours that I was used to. So we set out to dispel the myth that MPC was only suited to slow process control problems and showed that it could even be applied to switched power electronics systems. Through this activity in parallel with a couple of other groups in the world, among them the group of Graham Goodwin, we started this era of “fast MPC” and contributed to the spread of MPC to just about every control application area.

I would never claim that in the mentioned areas we made the most significant contributions and some of the results may even seem trivial to you now, but we were there at the beginning. The Hungarian author Arthur Koestler remarked that “the more original a discovery, the more obvious it seems afterwards”

Not withstanding this over-the-hill award that I received today and the mandatory retirement age in Switzerland I fully intend to strive to match these contributions in the coming years – together with my students, of course.

I want to close my remarks quoting from an interview Woody Allen gave last year. When he was asked “How do you feel about the aging process?” he replied: “Well, I’m against it. I think it has nothing to recommend it.”