What is fine-tuning?

Fine-tuning is a process for optimising AutogenAI’s Language Engines for enterprise-level tasks by exposing them to company specific literature, much of which will be new learning material for a generalised Language Engine. Generalised langage engines are trained on a vast quantity of web crawl data and digitised books. But they will not have seen highly company specific literature that might include:

  • Annual reports
  • Company statistics and facts kept in Knowledge Management Systems
  • Internal communications
  • Marketing materials
  • Internal training materials
  • Bids, tenders and proposals

Training a Language Engine on this data enables it to incorporate that information into the text it generates. Additionally, this exposure enables the Language Engine to:

  • Emulate a companies’ tone of voice. This could be formal, motivating or factual for example.
  • Adopt a preferred language structure – For example using long descriptive sentences as opposed to terse sentences and/or using bullet points instead of more flowing prose.

Below are example responses generated by an untrained ‘general engine’, compared to one trained on an (anonymised) company’s specific literature:

Default untrained model
Fine-tuned model

The ability of a trained language engine to perform reading comprehension tasks [OpenAI paper – page 18 | Table 3.7] on text that it has been fine-tuned on, as opposed to merely referencing it verbatim, allows the information to be used creatively to answer other questions or to integrate what it has learnt into its other responses. This results in higher quality responses compared with search engines or word/sentence matching.

How does it work?

Large Language Models have been pre-trained on nearly the entire corpus of digitised human knowledge. This includes, Common Crawl, Wikipedia, digital books and other internet content. They learn the statistical associations between billions of words and phrases. They then use these statistical associations to produce new output.

The mathematics involved during the fine-tuning process is complex. At its core it involves adjusting billions of parameters within the language engine’s neural network – it’s silicone brain. We now have the computing power to facilitate such vast calculations due to advances such as parallelised computation. LLMs require enormous calculations to be performed and only modern supercomputers are able to perform these in reasonable periods of time.

A language-based neural network – which power LLMs – being trained and updating its parameters.
Credit: Denis Dmitriev

Application-specific fine-tuning

When training a specific rather than a general language engine, a balance must be struck between fine-tuning the Language Engine too much (high exposure) which results in the engine over representing the new literature in its outputs and mutes its creativity, vs fine-tuning it too little (low exposure) resulting in outputs remaining too general. underweighting the literature in its outputs and remaining too general. AutogenAI’s team are specialists in achieving high-performance fine-tuning.

Once a Language Engine has been developed it can be continually updated and evolved with new data, expanding its knowledge base. It is also possible to create different versions of a particular Language Engine for particular applications. For example, a general Enterprise Language Engine might be trained on the entire corpus of an organisation. That company-specific Language Engine could then be further trained to generate marketing collateral (one version), or sector-specific bid-writing (further version or versions). Training rounds (referred to as ‘epochs’ by fine-tuners) for a specialist bid writing Engine might include:

  • Successful bids for the Engine to understand the most desirable answers and language
  • Unsuccessful bids for the Engine to understand less desirable answers and language
  • Publicly available invitations to tender (ITTs) to learn general language and structure
  • Specific ITTs that bid teams are responding to, to learn the commissioner’s langage and requirements
  • Sector and or contract-specific collateral providing relevant examples to model responses on

This means that business development teams could benefit from one language engine tailored to justice sector bids, and another specialised to facilities management for example.

As shown earlier in this response, the difference in response quality between a default “vanilla” Language Engine and one that has undergone fine-tuning is stark, giving those who invest in their fine-tuning a significant edge over those who do not.

Prompt Engineering

Even if an Engine has been fine-tuned this doesn’t guarantee high quality responses. You need to ask the right questions to achieve the response that you want.

Prompt engineering is responsible for the behind-the-scenes instructructions (‘prompts’) that are sent to the Language Engine each time a user clicks a button. Whatever the function of a particular button, it is a prompt engineer’s job to optimise the instructions given to the Language Engine. Good prompt engineering dramatically improves response quality and allows users to get the best out of their Engines.

At AutogenAI we believe it’s important to build an intuitive interface between the user and the Language Engine, and recognise that the supporting infrastructure may need to vary for each of our clients to best meet their needs. That’s why we were the first company in the UK to hire dedicated Prompt Engineers.

This is an example of the output produced by our ‘Add Case Study’ button:

“Add Case Study” button
What the user sees: Everything is done to optimise user experience, so fetching a case study can be done with a click of a button.

Behind the scenes of each button are a series of prompts which analyse the text and calculate the best methods to create the highest quality output for the user. The user is totally unaware of it by design, only seeing the end result of the prompt engineering.

This graphic ilustrates a simplified prompt engineering flow for an “Add Case Study” button (in practice prompt engineering can be complex, involving several steps and calls to numerous Engines).

prompt-engineering.png
A simplified prompt engineering flow

Step-by-step prompt engineering

  1. Articulation: Prompt engineering first determined that the existing text lacked the necessary detail to use it as a query for a quality case study, so first passed it through an articulation or expansion step.
  2. Internal Engine Response: Queries can be processed though many fine-tuned Enterprise Engines, so the prompt engineering  instructions decide which one is best for a particular task. The response is then used for the next steps.
  3. Examples (One-Shot): One-shot learning involves providing a single example of what we expect a good output to look like to the Engine. This primes the Engine to deliver the best response.
  4. Specific Query: Using a precise query and explicitly stating the assumptions that might have been implicit within the user query helps to deliver the final response.

Conclusion

Fine tuning allows companies to create their own bespoke Language Engines, trained on company data and engineered for their specific needs – something that will soon become industry standard practice but for early adopters will represent a seismic economic advantage over their competition! Further to this, prompt engineering has emerged as a high-impact specialism which is poised to transform the industry as Large Language Models are adopted by businesses and become mission critical tools within each corporation.