AI 安全：倫理學家與軟件工程師談模型幻覺

A quick note before you read: I wrote this in 2023, and life got so busy that I didn't open this blog again until today. I came back to publish something about AI and coding teams, and the first thing I saw was this unpublished draft.

I'm sharing it because of how extraordinarily relevant it still is. It's something we are actively combating. Yes, we need to be responsible. No, most of us aren't -- and it's mind-blowing that this is still out there. We're leaving it up to our tools to be safe, yet we're still the ones operating the controls. It's an incredibly beautiful dance we find ourselves in, and I wanted to share it.

We are in a tech turning-point in history: from one day to the next, AI has changed the game in most every software market. OpenAI's Chat software reached 100 million users within 2 months of deployment. This same audience base took Twitter (now X) 2 years to achieve. AI is everywhere and there is a race to integrate it into every aspect of software on the planet, and I can see why. Its potential to help humanity and the planet are tremendous: consider Aza Raskin's work to use AI to understand the language of animals, or using it to develop a way to eliminate micro-plastics from the ocean (Ocean Cleanup is doing a great job with Macro-Plastics, anyone know of micro-plastic initiatives?). While we are enthusiastic, it's crucial to remember this tech is new to public access. We are in uncharted territory and it is wildly exciting. However, this also means we must proceed with caution.

Don't ask me, I just work here.

I am a software developer, and have used APIs for diffusion models and LLMs to integrate AI into my apps. The process is relatively easy. I do not have formal training in math or data analysis, yet just the other day I built out a computer vision model. It took less than 10 minutes.

I am telling you this to illustrate the ease of application for this mind blowing technology.

This blog post is mostly opinionated, and addresses one big issue I have encountered with AI deployment: confidently wrong answers. There is an ocean of safety concerns surrounding AI, but to get the conversation rolling I chose this as an entry point.

This post is for anyone in the industry to contribute to, and I want your comments. I see industry professionals and academic researchers delving into the safety and ethics of AI. Simultaneously, developers are being asked to integrate AI without a clear roadmap. I believe there should be a stronger partnership between these groups to improve AI deployment in product development.

AI isn't new.

According to some pretty great brains, our first interaction with AI on a massive scale was the development of Social Media search algorithms that were optimized for engagement. LLMs have been in the works for years, and have finally reached the point of mass deployment. Hallucinations are a known issue. Although there have been improvements to decrease their occurrence, it is in the nature of the tech to happen. A LLM is not aware of itself (at least not yet), so it doesn't have the capability to check for accuracy. An LLM essentially consists of dimensions upon dimensions of interconnected nodes, and the process of finding the right answer involves lighting up areas of these nodes that find the best match to your question, then returning the result. Sometimes the best match isn't the right match, and unfortunately, one wrong answer feeds into subsequent wrong answers.

The AI Race

Hallucinations are a fact, but the excitement and the potential for the tech outweigh this downside. We know of the Big Dogs who have put a ton of time, money and compute power into developing LLMs: OpenAI and Google, to name two closed-source for-profit companies. You can use their API to interact with their models. Open source companies like Hugging Face have recently deployed models (think Falcon) and their aim is to democratize the tech so everyone has access.

The point is, the race of the Tech Giants continues. And the race of start-ups that rely on their LLMs is on, too.

What does that mean for us as software developers?

Based on how easy it is to use this tech, I can imagine developers will be asked to start using LLMs in their development process. I just spoke with a friend who is a software engineer, who is working on a project within his fintech company using external APIs to incorporate AI into payroll. I anticipate this becoming a standard request for developers. Upon hearing such news, I find myself astounded. As software engineers, we know that our job requires learning new tech quickly and often, but this is next level. To paraphrase Giada Pistilli, Principal Ethicist at Hugging Face, we are in the Wild Wild West of deployment and are being asked to do something that has never been done before--ever.

SOOO....what IS the problem?

My concern is not with the technical abilities of software engineers and developers: we pride ourselves on quick and efficient technical mastery. My concern is with the ethics: example, say we have successfully integrated AI into our fintech company to handle all things payroll and the product is deployed: how do we account for the hallucinations that will inevitably happen? Wrong answers, wrong tax reporting, wrong employee payout. These presumably will be anomalies, but one bad hiccup is enough to put a company under and cause an individual an unnecessary amount of stress.

That example was for a very black-and-white type of situation. What about the startups looking to use AI in the form of a chatbot therapist (at Boulder Startup Week 2023 I observed a few startups interested in this)? Now we are getting into the land of unclear distinctions. As we know, people are not numbers and are very complicated. What if a chatbot has a hallucination in the face of a human with a very real need for dependable information? This could cause devastating effects.

Accounting for error: a three-tier step

Level One: Big Tech Accountability

This one is giant. We want to trust the products they make and deploy them to stay competitive, and give the customers our company serves the best and latest tech. But first and foremost -- always, always do your research. For example: how does XYZ company who holds the rights to the LLM handle accountability? Are they projecting a big change in the future which might affect your software? Read how they are handling hallucinations and user unpredictability.

Level Two: Startup Accountability

So your startup has decided to use AI? Great! What are the safeguards in place to protect your company when a hallucination goes unnoticed and is used in exactly the wrong way?

Level Three: Personal Accountability

As consumers of generativeAI content, it is up to us to decide if we will trust the tech or do our own research. This is tough, because we are buying the software to make a process easier, and we do not expect to need to babysit something that is guaranteed to work. But for self-preservation's sake, perhaps we should. Checking our own records is always important, and reaching out to a human for verification when we come across a fishy piece of data is important too. Informing your customers of what to watch out for and promoting awareness is practicing good transparency.

Solutions for Developers

While I may not have definitive answers for us, I have questions that I hope can get the ball rolling in this conversation. In this spirit, let's share ideas for how to safely deploy AI tech, to keep consumers and companies alike in the safe zone.

First, it is important to recognize that developers are not the only players in the vast field of AI deployment. Aside from the coding and direct tech development, there are other critical stakeholders involved: project managers, the CTOs, the business stakeholders, the quality assurance testers, devOps team, and users providing feedback as a last line of defense. Some companies have more, and some have less. The point is, these roles together create a holistic framework in which responsibility for the success and ethical use of AI is distributed. As a developer, you have friends in the business who all have the same goal as you.

In the case where all these filters fail, and as software developer you are faced with some particularly fishy implementation of AI: is it above your pay grade to question? As developers, what is our level of responsibility to human ethics in this mad dash to AI? Creating software is a giant job, and already we juggle a lot. Practically speaking, do we have the time and energy and knowledge to be aware of the potential pitfalls of the AI software we create?

Thank you for your interest in this topic. As a developer deploying AI software or an individual in the field of AI Ethics, I would love to hear from you. How should a startup deploy AI Software in the context of hallucination accountability?