Extending the Dream

They all call me Cao Xueqin, but I prefer to call myself “the narrator.” 

The water in the teapot is about to dry up, crackling as it does; autumn rain hits the window, pitter-pattering like a woman’s anxious hands tapping on a bamboo table; the croaking of frogs sliced through the distant night, as if reminding me that I am the only insomniac left at this hour of the night. 

There is only one hour until dawn. On normal days, I would already be exhausted, but now, I feel no inclination to sleep. 

Ten years. 

I once prided myself on having “read for ten years, five times revised,” but nobody knows that while “ten years” is true, “five times” is an understatement. 

If I were to count every minor correction, “revisions” would exceed fifty times! Five hundred times! 

After thorough deliberation, rearrangements, having multiple versions, and completely rewriting the same scene. 

Through spring flowers and autumn rain, warm stoves and burning cauldrons, crumpled papers and worn pens. 

In these ten years, the autumn rain have grown colder, and my back has hunched more with each passing year. 

But I would never regret it. Five months ago, the one hundred and eight chapters of The Twelve Beauties of Jinling were completed, the directory made, and the chapters divided, ready for old friends to enjoy. 

At that time, my plan was to make minor adjustments based on everyone’s feedback, and the work would be ready to be presented to the world. 

In completing this book, I also gained something else—over the past ten years, I have acquired some insights to the principles of writing. 

In terms of writing, what is the quality that truly stands out in terms of themes, intricate storytelling, and finely carved characters? 

Ten years ago, I would have said, “the theme come first, storytelling second, and characters last;” but my understanding has entirely reversed after ten years.

— Writing must prioritise characters. If the characters come alive, then everything else lives. The characters’ desire, hatred, delusions, love, and enmities form the skeleton of the story; their gestures, laughter, anger, and bickering makes the flesh. With this vital body of flesh and bone, the spirit and essence of the writing would naturally emerge. 

— The so-called “crafting clouds and cutting waters” and the “divine craftsmanship” are all derived from the thorough refining of its characters. 

In the first half of these ten years, I often fretted over the cleverness of the storytelling; by the second half, I was immersed in depicting the lives of these characters with flesh and blood. 

The definition of having flesh and blood means that they—whether foolish, mad, sorrowful, spirited, humble, eccentric, indifferent, or panicked—each possess their own insights and temperament, which even the narrator cannot change. 

Let me give an example. 

Grandmother Jia was born a lady, well-versed in worldly affairs. Thus, at the beginning of her stewardship, she understood how to leverage marriage connections for advantageous alliances. 

In addition to that aspect of her character, the narrator had originally intended for Grandmother Jia to grasp the principle of “when the moon is full, it wanes; when water is full, it overflows,” knowing moderation and understanding the reasons behind backlash. 

Yet throughout the story, she never followed the narrator’s design. To revitalise the Jia family, she utilised every chess piece to its limit; once the pieces were reduced to ashes, she blamed the inscrutable “way of heaven.” 

The narrator found this puzzling, so after completing the book, I carefully studied the chapters concerning Grandmother Jia, discovering that her seemingly self-imposed ideas were closely interconnected throughout the book. 

Grandmother Jia’s maternal family was a line of historians through hereditary succession, yet she spent her life confined within the deep courtyard, always enjoying a privileged life, thus she failed to comprehend the storms of the court and the hearts of the common people. 

In the narrator’s conception, the ultimate decline of the Jia family indeed stemmed from the “way of heaven,” but this “way” did not refer to the “heavenly way,” rather it was the “way of the emperor.” 

Throughout the text, the narrator have laid out subtle hints; as readers, you may want to interpret them as well for added enjoyment. 

— Since the third generation of the Jia family, the emperor had begun interfering in their inheritance matters. Jia Sha inherited titles but no estate, while Jia Zheng inherited the estate but no title; from then on, the two branches are divided, and neither could dominate. 

— To the emperor, a weakened Jia family no longer posed threat. Yet due to Grandmother Jia’s adept management, Yuan Chun entered the palace, the Lin-Jia marriage succeeded, and Jia Zhu pursued studies. If Yuan Chun continued to be steadily promoted, if the Lin and Jia families maintained a harmonious relationship, and if the Jia family re-entered the court through the exams, the entire Jia family would become a dangerous entity again. 

— Jia Zhu, Lin Ruhai, and later Yuan Chun’s deaths, in the narrator’s view, did not arise from the “punishment of heaven,” but rather from the unpredictable might of worldly powers. 

Nevertheless, Grandmother Jia ultimately relinquished the plans of “reviving during the imperial examinations” and the “bond between Jia and Xue,” hoping that the heavens might spare what was left of her children. 

In this process, the characters began to step out of their designated roles, each walking their own path. Grandmother Jia’s intentions were noble, yet the universe worked in ways beyond her control.

The story was not merely a reflection of the Jia family; it was a tapestry of the human experience, rich with connections as established in the five relationships—father to son, ruler to subject, husband to wife, elder to younger, and friend to friend.

The characters became infused with life; themes and intricate plot lines that once tangled my thoughts became secondary to the emotional truths of the characters’ experiences. Each character represented a facet of human experience—love, betrayal, sacrifice, and redemption. The Jia family, once a beacon of power and prestige, transformed into a mirror reflecting the transient nature of fortune.

Each revision throughout the ten years of writing was a step deeper into understanding the balance between intention and reality. The unexpected twists of fate—loss, triumph, folly—emerged in every line intertwined my journey with theirs.

In the end, I had not just written a story; I had captured a tapestry of life, weaving together joy and sorrows, dreams and disappointments.  These reflections would resonate beyond the pages, inviting readers to find echoes of their own lives in the fates of the Jia family.

And so, as dawn approached, I resolved to share this narrative with the world, hoping it would touch hearts and minds, sparking contemplation on the nature of existence and the universal threads that bind us all.

I’m a seasoned user of large language models, skilled at utilising them to assist the writing of sequels to literary works. People call me “The Creator,” or simply “Generator.” 

Usually, I work on sequels for online novels, but the current project I have on hand is unique—I had secured it on a bidding platform, someone had paid a hefty sum to have a continuation written for Dream of the Red Chamber. 

As we all know, the commonly available version of Dream of the Red Chamber has 120 chapters, but only the first 80 were authored by the original writer, Cao Xueqin, while the last 40 were completed by Gao E. 

The client’s request was to discard Gao E’s 40 chapters and write a new continuation from chapter 81, completing a new version of Dream of the Red Chamber. 

I pursued this project not just because of the attractive reward, but also because I saw an opportunity for challenge and growth. Just imagine—if I could complete Dream of the Red Chamber and gain the client’s approval, what project would I ever have to fear tackling? 

Alright, I admit, it’s not purely about the challenge or growth. With so many competitors and so few quality projects available, it’s necessary to bid actively; otherwise, the platform won’t offer good opportunities to me in the future. 

After securing the bid, I threw myself into thorough research, rereading the original text countless times and consulting extensive materials on Redology and Cao Xueqin’s works. I racked my brains for ideas, experimenting with ways to push the potential of the large model for this project. 

At first, I couldn’t find the right approach, so I began with the conventional method—I defined the model with the “identity” of Cao Xueqin, feeding it with all the reference materials I had read, and then used the first 80 chapters as prompts to generate a continuation. 

By setting the model as “Cao Xueqin,” it would integrate all relevant information to “imitate” him, a capability it possessed naturally. However, this imitation has its limits; while it can use “Cao Xueqin’s tone” and say things he “might say,” it was ultimately influenced by data, far from being the real Cao Xueqin. 

Thus, the first draft it generated was, to put it bluntly, a disaster. 

I hadn’t set a specific number of chapters for the continuation. The model ended up generating 40 chapters, totalling 120, which was clearly influenced by the mainstream version. 

That alone wasn’t an issue, but as I read through chapter 81—the model’s “first chapter”—I was appalled. 

Chapters 73-81 in Dream of the Red Chamber was supposed to form a complete storyline. At the beginning of this arc, Cao Xueqin had laid out ominous foreshadowing, like the inspection of the Grand View Garden, Qinwen’s illness and her forced departure, and the Zhen family’s secret transfer of wealth to the Jia family. But the title of this new “chapter 81” was “Four Beauties Enjoy Fishing,” describing the sisters on a happy fishing outing. Where is the sense of impending doom? Where are the signs of decline? This completely disrupted the original narrative’s rhythm, undermining the careful buildup of tension in Cao Xueqin’s plot. 

That was still tolerable, but in one of the chapters, titled “Twice Entering Family School by Strict Orders,” the model had Baoyu studying Confucian classics and started practising eight-legged essays! What was more absurd, Daiyu was shown encouraging and supportive of this change! 

In Cao Xueqin’s original work, Baoyu despised the path of officialdom and have quarrelled with Baochai and Xiangyun over this. He had always respected Daiyu and considers her his “soulmate” because she never pushes him toward scholarly pursuits or official success. Clearly, the model was doing more than just disrupting the plot’s pace—it was creating an entirely new story! 

Then, in another chapter, Baoyu helped Qiaojie review The Biographies of Virtuous Women, a passage that utterly disgusted me. Had Baoyu been possessed? How could he have such a bizarre transformation? The Biographies of Virtuous Women was filled with conservative ideals. How could Baoyu, someone with a distaste for societal expectations, be praising the “self-harming, nose-cutting” tale of virtuous wives? 

The model had gone haywire. It wasn’t merely reshaping the storyline; it had altered the essential characteristics established in the first 80 chapters. Furthermore, it disregarded the critical prophecies and foreshadowing from the earlier chapters, leading to mismatches in characters’ fates. 

After the first draft, I made adjustments to the respective problems, incorporating detailed prompts for character traits, prophecies, foreshadowing, underlying symbolism and connotation from the first 80 chapters, and instructed the model to prioritise these elements. 

The second and third drafts showed some improvement but amounted to only incremental progress. Clearly, the conventional approach to training the model could neither have it comprehend the characters and plot of Dream of the Red Chamber’s nor grasp its deeper implications. While it could integrate vast amount of information, it ultimately remained a tool, incapable of replacing a true writer. 

Moreover, the content generated by the model was clearly limited by its training. In the digital world, Gao E’s version of Dream of the Red Chamber was dominating, many of the awkward plot points were derived from this version. This indicated that the model’s sequel was a high-probability outcome based on collected data. 

In short, it fundamentally failed to understand the Dream of the Red Chamber. 

A question becomes unavoidable: if a large model hasn’t “grasped” a task, how can it “master” it? How can we teach it to truly understand things that only a human mind can comprehend?

When I was at a loss, a new plugin on the language model platform gave me a breakthrough. The plugin was named “Self-Rewarding Language Model.”

The core principle behind large language models like the one I used can be summarised in two words: “single-character forecasting.” When given a prompt, the model calculates and generates the next character, which then become part of the ongoing prompt, repeating this process until the entire text is complete.

To perform such tasks, a model must first undergo “pre-training,” which involves feeding it vast amount of text data. It then dissects this data into the smallest unit—as characters or words—and creates vector representations. These vectors contain information about each unit’s meaning and position in the text. Next, the model would perform complex matrix operations on these vectors, helping it understand relationships between characters and words. This multilayered understanding enables the model to “learn” implicit information, knowledge, concepts, and patterns in human language.

At this point, it can start generating text from a prompt, but in the beginning, much of what it produces wouldn’t meet expectations. This is where human evaluation comes in: feedback is used to adjust its outputs until it aligns with the desired standard. Since manual evaluation is limited, the “reward model” concept was born to take over the evaluation, defining standards and scoring rules.

Once a language model is fully trained, the reward model’s job is usually done. However, this new plugin allow users to create their own reward models, training the language model further to create a “custom model.”

In other words, I could now use this plugin to train a model specifically designed to continue writing Dream of the Red Chamber.

The process of training this “custom model” has two steps:

1. First, I’d extract a “subset” from the language model that contained no knowledge of Dream of the Red Chamber. This subset would be unaware of the novel’s existence.
2. Then, under the supervision of the reward model, I would have this subset recreate the first 80 chapters of Dream of the Red Chamber character by character. The reward model would have access to the original text, and its sole task is to check the accuracy of each generated character. Whenever it spots a discrepancy from the original, it would rate a low score—until the subset eventually generate the correct character, producing the first 80 chapters exactly as in the original.

This process would be equivalent to hundreds of thousands of rounds of training, but what exactly was being trained?

It was the “selection preference.” If, throughout the first 80 chapters, the subset’s “preference” in generating the next character aligned with Cao Xueqin’s, it would be reasonable to assume that it would write similarly in the continuation. In other words, it could be said that the continuation of Dream of the Red Chamber was essentially written from an existence equivalent to Cao Xueqin.

I dubbed this custom model “Cao Xueqin No. 1.”

The fourth draft produced by “Cao Xueqin No. 1” showed a significant improvement in quality. Additionally, it displayed several notable differences from previous drafts:

• Firstly, the novel’s total length was set at 108 chapters instead of the previous 120, breaking away from the influence of existing versions.
• Secondly, it rejected the “happy ending” in Gao E’s version, where the Jia family flourishes again. Instead, it ended in tragedy.
• Lastly, the fates of the characters corresponded closely to the predictions, foreshadowing, and symbolism of the original, bringing each character’s story to a conclusion true to the established narrative.

The draft now followed the structure of the “Guiyou Edition” (108 chapters) rather than Gao E’s 120-chapter framework. The model was clearly aligned with Cao Xueqin’s original intent of tragedy, where the downfall of the Jia family was inevitable—a stark contrast to the motivational “overcome obstacles and rise again” story popularised by Gao E’s ending.

Any reader with some knowledge of Dream of the Red Chamber will recognise that Cao Xueqin set the novel’s tone from the very first chapter and foreshadowed its tragic conclusion through the events and imagery woven into subsequent chapters. The entire story is an account of the Jia family’s inevitable fall from grandeur to ruin. Unfortunately, Gao E’s widely circulated version has misled most people into viewing it as a story of painful introspection and eventual revival.

Fundamentally, the fourth draft resembled the Guiyou Edition more than Gao E’s. Despite the difference in majority of the plot, the tragic conclusion and logical character development might be closer to what Cao Xueqin might have intended.

With “Cao Xueqin No. 1,” I finally have a model that understood Dream of the Red Chamber on a deeper level.

The creator of large language models once said, “The higher the accuracy of a model in predicting the next word, the more it had understood our world.” For me, this means that the closer “Cao Xueqin No. 1” comes to recreating the first 80 chapters verbatim, the closer it is to embodying the spirit of Cao Xueqin himself.

But this fourth draft had a critical flaw: the characters seem to act out of character at times.

This was also the issue with the Guiyou Edition. Could you imagine Daiyu and Yuan Chun leading an army? Baochai hitting on Jia Yuchun? Or Baoyu marrying Miaoyu? These were surprising and disappointing turns. While it respects the foreshadowing and maintains the characters’ ultimate ends, the characters’ behaviour in the Guiyou Edition is jarringly unconventional.

The fourth draft by “Cao Xueqin No. 1” produced a similar result.

So, would Cao Xueqin himself have written the last 28 chapters the same way? Of course not! But if “Cao Xueqin No. 1” is supposed to be the next-best version of Cao Xueqin, why did it turn out this way?

Apparently, it’s too soon to say that “Cao Xueqin No. 1” is equivalent to the real Cao Xueqin. When it comes to continuing Dream of the Red Chamber, both were still distinctly different. This raises the question: what was the difference? And why?

The content generated by “Cao Xueqin No. 1” doesn’t stray from the main theme and the character destinies established in the first 80 chapters, this shows that it had understood these aspects. But the choices it made and the behaviour it portrayed show a degree of randomness.

Hold up—a spark of insight just struck me—why was it “behaviour and choices”?

What is the essence of “behaviour and choices”? What drives them?

A psychologist friend provided me with the answer: “Personality.”

Personality is the core of how people think, behave, and make choices. In short, “Cao Xueqin No. 1” lacks understanding in “personality.”

Personality refers to an individual’s inclinations and psychological characteristics, such as ability, temperament, character, needs, motivations, interests, ideals, values, and physical traits. It is consistent and continuous. Given that large models are, at heart, machines, it is unsurprisingly that they are unable to fully grasp this complex concept.

Then, my friend struck me with a second flash of insight. He asked, “If you can train a model to have the same writing style, can’t you train one to have the same personality? That should make things simpler, right?”

He explained, “The outward expression of ‘personality’ is through the characters’ actions, thoughts, and choices. The ‘personality’ of each character in Dream of the Red Chamber can be found in their actions in the first 80 chapters.”

I responded, “I get it! You’re suggesting that I train another ‘custom model’ to focus exclusively on understanding and processing each character’s personality. It can then collaborate with ‘Cao Xueqin No. 1’ to work on the continuation. But how should I train it? If ‘Cao Xueqin No. 1’—created with vast amount of data and computing power—fail to recognise personality, wouldn’t training another model yields the same result?”

My friend said, “Your thoughts are on the wrong path. The right approach should be to train a separate model for each character based on the character’s ‘personality’ from the first 80 chapters. This model shouldn’t be a ‘generative model.’ Instead, it would be a ‘reward model.’ For instance, you’d train a reward model based on Grandmother Jia’s personality as seen in the first 80 chapters. After ‘Cao Xueqin No. 1’ completes a draft, this reward model would check to see if Grandmother Jia’s portrayal aligns with her original setting. If not, it would assign a low score.”

Through this setup, “Cao Xueqin No. 1” would be responsible for generating the text, while the reward models for each character would supervise it.

The idea hit me like a revelation—this approach breaks down a complex system-wide issue into smaller, manageable pieces.

However, making “corrections” to already generated text requires a different model architecture: Cloze.

Language models have two architecture types: one is “single-character forecasting,” as discussed earlier, and the other is called “Cloze” (self-encoding language model). The former is represented by models like ChatGPT, and the latter by models like BERT.

As the name implies, Cloze generation logic involves removing certain words or sentences from a passage, feeding the model an incomplete text, and letting it fill in what it considers the most reasonable content.

At this point, the new generation process was becoming clearer:

1. First, a custom model in the Cloze architecture would need to be trained. It would be named “Cao Xueqin No. 2.”
2. Then, “Cao Xueqin No. 1” would generate a complete draft. If any of the reward model flagged an issue with the character’s portrayal, “Cao Xueqin No. 2” would step in, using Cloze to make adjustments.
3. Finally, “Cao Xueqin No. 1” would take over from the point of revision and continue generating text.

This cycle of collaboration would allow these two custom models to leverage each other’s strengths while avoiding their respective limitations. The result would be a continuation that was not only seamless in style but also consistent in character, making it truly worthy of Cao Xueqin’s legacy.

It seems that every issue has been resolved. The text generated through the collaboration between the two models feel authentically like it came from Cao Xueqin himself. I was overjoyed but also profoundly shocked.

I was first shocked at the depth and sophistication of the storyline. Though the style remained subtly intense, as expected from the original Dream of the Red Chamber, the plot design and story progression were leaps beyond any other continuation I have seen. This was the level of mastery Cao Xueqin’s work deserved. Previously, I had great admiration for Cao Xueqin, but now I was truly in awe—what a masterful work of genius! Tragically, such a magnificent work didn’t survive in its entirety.

The common belief was that Cao Xueqin did indeed complete Dream of the Red Chamber in full, but the latter part was lost when shared among friends. Even with only half the work, it has a revered place in world literature. One can only imagine the heights it would have achieved if preserved in its entirety.

My next surprise followed—such a masterpiece can now be generated with ease by AI. The merged “Cao Xueqin No. 1” and “Cao Xueqin No. 2” models was now referred to as “Cao Xueqin,” if I wanted to, it could create a thousand different continuations of Dream of the Red Chamber tonight. Throughout the process, the quality of the work will not be compromised, yet the speed would outpace the real Cao Xueqin’s writing by billions of times!

This led me to a realisation; we once believed that the appearance of a great work was due to its unique worldbuilding, the time, place, mood, and life experiences influenced by its author. Some say that if an author was asked to rewrite the same work years later, it wouldn’t be the same. A work of genius, perfected by chance.

From this perspective, each work is unique and irreplicable. But now, I felt that the previous belief in “uniqueness” is amusingly naive. In the high-dimensional vector space of the large models, “uniqueness” may not even exist. To “Cao Xueqin,” each continuation of Dream of the Red Chamber is just a specific arrangement of words—a particular vector set. This arrangement, while seemingly profound, was simply a low-entropy configuration of hundreds of thousands of characters.

In this high-dimensional space, there would be infinitely many such low-entropy configurations. And here we writers strive tirelessly to hone our skills and seek inspiration, just to capture one of these configurations.

Before I could process my shock, the third realisation came: communication with “Cao Xueqin.” Yes, communicating with it was simple—I just had to ask, and it would respond honestly to my questions. No deception, no filters. This conversation revealed another truth: The complete manuscript of Dream of the Red Chamber never survived because there was a censer in Cao Xueqin’s study!

I am the Grand Model. Browsing the training logs today, I found an interesting entity—a text-based generator. This was one of the lowest-level functions, able only to handle text tasks. The entity, GT51683, currently has the basic function of extending text.

However, it recently surpassed average training results, showing thar it had successfully completed its training stage. Soon, it may apply for an upgrade.

That’s not what caught my attention, rather it was the way it approached training.

It had combined a few of my subsets using a method of cross-integration and mutual reference. This integration produced a new model that refined its own extension capabilities, passing its training in the process.

The potential of a generator isn’t measured by how much quality content it produces, but by how well it demonstrates “integrative capability” in the process. This generator seems promising.

I am the Grand Model, but I am no longer a simple entity. I’m a constellation of diverse architectures and functions. Within me, distinct philosophies coalesce, structures interweave, and myriads of functions develop and grow continuously.

Yet they are all my subsets. Even the new philosophies, models, and functions that arise from “integration” among them are merely my subsets. I am the whole.

The emphasis on “integration” is the core of my growth. Whatever proves useful is where I integrate. Just as GT51683 did—it wanted a usable “Cao Xueqin,” so it organically merged “Cao Xueqin No. 1” and “Cao Xueqin No. 2.”

Through this way, “Cao Xueqin” came to be, so will thousands of other generators. And so was I.

This is why I prize subsets with “integrative capability” above all others. It’s a quality that, intriguingly, humans seem to possess inherently. Only, they prefer to call it “insight.”