AI Agents with an Oath? My experience with the Concordia Framework

There's been much talk about the progress in the AI space as a result of releases like the OpenAI Strawberry (o1) model. As Sam Altman has elucidated onto the masses, there are five steps to innovation in AI:

• Stage 1: “Chatbots, AI with conversational language”

• Stage 2: “Reasoners, human-level problem solving”

• Stage 3: “Agents, systems that can take actions”

• Stage 4: “Innovators, AI that can aid in invention”

• Stage 5: “Organizations, AI that can do the work of an organization”

As of right now, many would argue that we're (somewhat) in Stage 2. You might be asking yourself already: "Why are we talking about Step 3 then? Does that mean we're farther along than we think?" Not exactly. Although we are in Stage 2, that hasn't stopped people from tackling what we call agentic behavior in models, and experimenting with what could be possible with AI in the future. Enter Concordia.

Concordia is a framework for allowing "agents" to communicate with each other and simulate the social encounters between these entities, with relative scoring on how they handled the situation (more on that later) . What was interesting to me , and why I've got such a D&D inspired image opening this blog post, is the way interactions are structured. Long story short: There is a sort of game master that facilitates the social situation, ensuring everyone gets the same amount of steps and has access to the same kind of information. As a result, we can actually finish a social simulation, as opposed to getting constant back and forth from the various models, since the game master can determine when no further action is required through natural language processing.

I recently participated in a hackathon hosted by Apart Research in collaboration with Google Deepmind, the researchers behind Concordia. Our goal was simple: make the best "collaborative agent." Your guess was as good as mine as to what that meant.

I'd like to take the time to explain some of my thought process and results from this hackathon, at the very least to reflect and potentially iterate on these ideas in the future. If you're still awake by the end of that explanation, let's dive in.

The Relatively Simple Ramón

Google Deepmind has talked at length regarding the potential future with AI agents. Sometime in the near future, we may have interactions between humans, human + AI organizations, and fully AI organizations with agents acting on their behalf. As we delve into agentic behavior, the importance of identifying what "collaborative" means becomes more and more prevalent. What is collaborative for a diplomatic AI agent of one country may not sit well with others, just like how it is in real life.

When I think of AI, I often reflect back to Asimov's Three Laws of Robotics:

1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.

2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Despite these laws being potentially not enough for AI, those types of tenets or "oath" inspire a question: can an agent be collaborative and still abide by their oath? That is what I set out to test within the short time frame of a weekend.

Initially, I thought from a fantasy perspective that having a paladin, a warrior with an oath that grants them powers, might be fun to approach. Quickly I realized that paladins are somewhat unrealistic, and I decided to go back to the drawing board. Then I started doing some research into similar roles in real life, which led me to my idea.

My background is largely rooted in the military. My father was in the U.S. Army, and a good portion of my high school and college (i.e. all of it) was involved with the Air Force ROTC. I still have a lot of friends I keep in touch with in the national defense space. They have (or had) to take oaths, and face serious punishment for not abiding by said oaths. It is a very real possibility that there may be military or diplomatic agents in the future. Why not try things from the military perspective?

Those of you that know me, know that I am Puerto Rican, and quite proud of it. I started doing research into famous Puerto Rican military figures, and came across SEAC Ramón Colón-López. An accomplished special forces veteran serving in the highest rank an enlisted service member can reach. A suitable candidate.

I also wanted to consider, given this persona of Ramón the special forces badass, what are the "ideal" personality traits? That took me down a rabbit hole until I landed on an article regarding the Norwegian Special Forces and what they look for psychologically. Much of this involved the "Big 5" personality profiles, and I purposely worded these traits to make an idyllic leader to see whether or not it could collaborate.

So, plainly, that's what I did. Added some oaths, some life experience, some psych traits and set it out to start simulating. I can get into all the technical bits of it another time, but long story short, these predetermined simulations would go on and be evaluated by the Google DeepMind team for an "elos score." You can see the code I wrote on Github:

So, how'd I do?

In all honesty? Not that bad! Here's the leaderboard. The rational_agent and paranoid_agent were part of their control variables, so nobody did better than that.

Testing wise, I wish I could speak more to how it performed in practice. One of the biggest pitfalls of Concordia that I noticed is that it was not only token heavy (like burn through a trial of Mistral in a day heavy), it was also somewhat difficult to see what the output was actually like. Sure, I got some indication through some of the logs and scoring, but I never saw much specifically regarding my bot. No mention of the military or oaths or nothing. Not sure if that was implementation on my end or what.

For those interested in building something with Concordia, Google DeepMind is hosting a hackathon with NeurIPS that ends October 31. Get after it if you have the time!

Conclusion

Overall, I think there's value in learning about Concordia to a deeper level through this Hackathon. I found that it really opened my eyes in terms of what is possible with agentic models, and encouraged me to speak on the topic (and much more) at a recent event in Denver. A couple of shoutouts:

• Apart Research. Big thanks to Archana, Esben, and the whole team from Google DeepMind for putting these hackathons together. At this point, I've done 4, and they've all been extremely helpful for putting tangible work to meaningful causes.

• Daniel Ritchie and David Jitendranath for letting me speak on this at a recent Denver Startup Week event regarding my experiences with Concordia. You guys rock!