Quick update on Tom's progress

To plan the modelling phases of Tom's process, I initially targeted the interaction between Tom and myself, which automatically narrowed my focus to Large Language Models (LLMs). The simplest case of real-time user interaction is a chatbot, and I thought this would be beneficial because i.) validating an experiment is far more trivial when you have face-to-face interaction with the agent, ii.) I wanted to target the area with the sparsest data, since I had predicted it would be one of my greatest challenges in achieving what I want, and iii.) it gives me the flexibility to choose between modelling methods like probabilistic, (non-)parametric models, graph networks and deep learning, if I am feeling confident on a Saturday, as on Sundays I have church sessions with Priest Mike Crotch.

Thinking of Tom's multi-model process as a spider web, the problem was defining a function at the web's knots. The frequent setbacks here involved rethinking how to structure the cobweb. In some cases, its structure depends on another cobweb's, and if you end up in a loophole, then that cobweb is the data collection centre behind the y-variable you are predicting. It was things like this that humbled me and made me consider throwing my laptop out of a two-storey building. To minimise these problems, I read old programming books (the same way I study for maths) and watched YouTube videos on how programmers do things, elegantly.

Back to Tom's process: on the NLP side, I have focused on features accessible in real time that can be incorporated into fine-tuning Tom's base LLM. Recall that the skeleton refers to Tom's multimodal connections between all his pre-trained models. What follows covers the language component of Tom's process, in terms of his skeleton and models. Previously, I was searching for a way to build a hooking module that attaches to the sub-layers of a neural network and transforms the weights in some way. But fear not: I am now aware of the existence of adapters (God's gift) and the Python library PEFT (God's way of gifting Eve to my Adam).

Tom's Policy Function

Figure 1 illustrates the weights assigned to the merged adapters used to fine-tune Tom's PEFT model, adapted from the initial network. The base model, as the language component of his skeleton, is instructed to role-play using Tom’s identity information. The same prompt is passed to the other adapter models, which differ only in their manner of speech and a few other conditions I am still reconsidering through more trial and error. Initially, I tried building a block that generates emotion-conditioned responses, called the Response space. However, given the complex nature of emotion labels, this component had to be rethought. That led me to opt for the second method: combining emotions with the dialogue learner as one cobweb (or focused task).

Tom's roleplaying as a curious agent would prompt the model to generate responses like Sherlock Holmes in a murder scene.

Figure 1: Tom's Fine-tuning Policy Network

Figure 2: Determining the coefficients to Tom's questioning states or responses.

The Clone model is fed and updated with responses written by me. Unlike the others, this adapter is fine-tuned in a sequence-to-sequence manner. To see how the weights are adjusted, consider Figure 2, which shows the weight given to curious responses at each optimisation step. Simply put, it is how much you want the LLM to respond with a question, given the context.