AI-generated Quick Answers in the MATLAB Documentation
Here at MathWorks, we are all very proud of the quality of our documentation. We put a huge amount of effort into it and it clearly shows! Most of our users rate MATLAB's documentation very highly and frequently cite it as one of the standout features of the product. Some even tell us that they prefer to learn topics from our documentation than from textbooks!
MathWorks has an army of documentation writers who work alongside development to deliver this amazing part of our products but almost every MathWorker has at least a small hand in it. We are all able to report documentation errors or make suggestions for improvement and they are taken as seriously as bug reports or enhancement requests for the software itself. The documentation is constantly evolving as we all strive to deliver the best experience to our users.
No matter how much we write, however, it is impossible to always cover exactly what every one of our > 5 million users might be looking for. Furthermore, as MATLAB grows, finding exactly what you need among the thousands of features becomes increasingly difficult. It would be nice to get a little extra help.
Introducing AI-generated Quick Answers
This brings me to our latest development: AI-generated quick answers. Head over to the MATLAB Help Center, click on the Search icon and search for something. I searched for 'How do I check my GPU compatibility with MATLAB?' and alongside the usual documentation search results I get the beginning of an AI generated response.
Clicking on 'Show more' gives me the full AI-generated response.
The AI has used the documentation search results along with my original query to form the response. We say that it is grounded by our documentation. I think of it as an on-the-fly remix of the doc pages that attempts to return information relevant to the query. Often there will also be unique AI-generated code that is relevant to your query.
The AI goes on to show the sources of the documentation it used to make its response and also offers you the chance to provide feedback regarding the quality of the response. Finally, there are some suggested 'Related searches' that the AI thinks will interest you.
That's it! That's the feature. I could stop there and move on to the next article but I'm guessing that, like me, you have a bunch of questions about this new development. Here is a summary of the Q and A I had with with development while writing this article.
Me: Gen-AI in the MATLAB doc? Aren't you worried about accuracy?
As with most current AI-related projects, this feature uses Large Language Models (LLM) as the underlying technology. These can be both very powerful and incredibly useful but, as you rightly point out, can be subject to accuracy issues. For example, they can 'hallucinate' incorrect information at times. This is an issue with all existing LLM-based technologies and is a limitation that one should always be mindful of. Even with this issue, LLMs are just too useful to ignore.
None of MATLAB's documentation is written by LLMs and that hasn't changed here. This is just an AI-generated summary of some of the documentation relevant to to your query. Maybe you'll find it useful, maybe you won't and you'll go-on to search through the rest of the documentation as you've always done.
To assist you, we start by clearly labeling the fact that this is an AI generated response and we remind you to 'Validate generated output before use'. We also show the most relevant sources of the actual documentation that were used to generate the response. As such, you can think of the AI-generated response as a suggestion of what to look for in the documentation itself.
Me: I get the same AI-generated result every time for any given query. Why?
LLMs are stochastic meaning that if you make the same query twice to an LLM model then you'll get slightly different answers. The reason why Quick Answers behaves differently is because we cache results for speed. The cached result currently persists for 30 days but this is subject to change at any time.
Me: I got an amazing response to a query! Can I link to it and post it on the blog?
You can link to the query but you can't guarantee that whoever follows that link will get the same response that you did. We cache responses for a short time but these are temporary. LLMs are stochastic by nature and we can change the model behind the scenes at any time so the response to a query will change over time.
Me: I got a terrible AI-generated answer to a query. Who do I complain to?
Click on the Thumbs-Down icon next to the query. This tells us that you didn't like it and you can give us more details about why you didn't like it if you want.
Me: What's the underlying AI-model?
Ever since we launched the AI-Chat playground, MathWorks has been developing its own AI infrastructure that's shared by various products and services including the playground, MATLAB Copilot and this feature: Quick Answers. We use LLMs running in the Azure OpenAI Service along with a RAG pipeline that is updated pretty frequently. Stating what model(s) we are using at any given time would soon become stale information.
Me: Do I need to be logged into my MathWorks account to get AI-generated Quick Answers?
No. Quick Answers is available to everybody who visits our site.
Me: So Quick Answers is free?
Yes.
Me: What if someone really dislikes AI responses? Can they turn it off?
Yes, just click on the X icon at the top right of the first AI-response you get and you won't get any more AI-generated responses for the rest of that session.
Over to you
Have you tried this new feature of the documentation yet? What do you think of it? Do you have any questions that I haven't covered yet?
- Category:
- artificial intelligence (AI),
- Gen-AI,
- New Features
Comments
To leave a comment, please click here to sign in to your MathWorks Account or create a new one.