Legacy Modernization meets GenAI

GenAI period: A timeline of key occasions

One major cause for the
present wave of hype and pleasure round GenAI is the
versatility and excessive efficiency of general-purpose LLMs. Every new technology of those fashions has constantly
proven enhancements in pure language comprehension, inference, and response
high quality. We’re seeing plenty of organizations leveraging these highly effective
fashions to fulfill their particular wants. Moreover, the introduction of
multimodal AIs, similar to text-to-image generative fashions like DALL-E, alongside
with AI fashions able to video and audio comprehension and technology,
has additional expanded the applicability of GenAIs. Furthermore, the
newest AI fashions can retrieve new info from real-time sources,
past what’s included of their coaching datasets, additional broadening
their scope and utility.

Since then, we now have noticed the emergence of latest software program merchandise designed
with GenAI at their core. In different instances, present merchandise have turn out to be
GenAI-enabled by incorporating new options beforehand unavailable. These
merchandise sometimes make the most of basic objective LLMs, however these quickly hit limitations when their use case goes past
prompting the LLM to generate responses purely based mostly on the information it has been skilled with (text-to-text
transformations). For example, in case your use case requires an LLM to know and
entry your group’s knowledge, essentially the most economically viable answer typically
entails implementing a Retrieval-Augmented Era (RAG) strategy.
Alternatively, or together with RAG, fine-tuning a general-purpose mannequin is perhaps applicable,
particularly for those who want the mannequin to deal with complicated guidelines in a specialised
area, or if regulatory necessities necessitate exact management over the
mannequin’s outputs.

The widespread emergence of GenAI-powered merchandise may be partly
attributed to the supply of quite a few instruments and growth
frameworks. These instruments have democratized GenAI, offering abstractions
over the complexities of LLM-powered workflows and enabling groups to run
fast experiments in sandbox environments with out requiring AI technical
experience. Nonetheless, warning have to be exercised in these comparatively early
days to not fall into traps of comfort with frameworks to which
Thoughtworks’ current expertise radar
attests.

Issues that make modernization costly

Once we started exploring using “GenAI for Modernization”, we
centered on issues that we knew we’d face many times – issues
we knew had been those inflicting modernization to be time or value
prohibitive.

• How can we perceive the prevailing implementation particulars of a system?
• How can we perceive its design?
• How can we collect information about it with out having a human professional out there
  to information us?
• Can we assist with idiomatic translation of code at scale to our desired tech
  stack? How?
• How can we decrease dangers from modernization by bettering and including
  automated assessments as a security internet?
• Can we extract from the codebase the domains, subdomains, and
  capabilities?
• How can we offer higher security nets in order that variations in habits
  between outdated techniques and new techniques are clear and intentional? How will we allow
  cut-overs to be as headache free as potential?

Not all of those questions could also be related in each modernization
effort. We now have intentionally channeled our issues from essentially the most
difficult modernization situations: Mainframes. These are a number of the
most vital legacy techniques we encounter, each when it comes to dimension and
complexity. If we will resolve these questions on this state of affairs, then there
will definitely be fruit born for different expertise stacks.

The Structure of CodeConcise

CodeConcise is impressed by the Code-as-data
idea, the place code is
handled and analyzed in methods historically reserved for knowledge. This implies
we aren’t treating code simply as textual content, however by way of using language
particular parsers, we will extract its intrinsic construction, and map the
relationships between entities within the code. That is completed by parsing the
code right into a forest of Summary Syntax Bushes (ASTs), that are then
saved in a graph database.

Determine 2: An ingestion pipeline in CodeConcise.

Edges between nodes are then established, for instance an edge is perhaps saying
“the code on this node transfers management to the code in that node”. This course of
doesn’t solely permit us to know how one file within the codebase would possibly relate
to a different, however we additionally extract at a a lot granular stage, for instance, which
conditional department of the code in a single file transfers management to code within the
different file. The power to traverse the codebase at such a stage of granularity
is especially vital because it reduces noise (i.e. pointless code) from the
context supplied to LLMs, particularly related for information that don’t include
extremely cohesive code. Primarily, there are two advantages we observe from this
noise discount. First, the LLM is extra more likely to keep focussed on the immediate.
Second, we use the restricted area within the context window in an environment friendly method so we
can match extra info into one single immediate. Successfully, this permits the
LLM to research code in a method that isn’t restricted by how the code is organized in
the primary place by builders. We discuss with this deterministic course of because the ingestion pipeline.

Determine 3: A simplified illustration of how a information graph would possibly appear to be for a Java codebase.

Subsequently, a comprehension pipeline traverses the graph utilizing a number of
algorithms, similar to Depth-first Search with
backtracking in post-order
traversal, to complement the graph with LLM-generated explanations at numerous depths
(e.g. strategies, lessons, packages). Whereas some approaches at this stage are
widespread throughout legacy tech stacks, we now have additionally engineered prompts in our
comprehension pipeline tailor-made to particular languages or frameworks. As we started
utilizing CodeConcise with actual, manufacturing shopper code, we recognised the necessity to
maintain the comprehension pipeline extensible. This ensures we will extract the
information most precious to our customers, contemplating their particular area context.
For instance, at one shopper, we found {that a} question to a particular database
desk carried out in code could be higher understood by Enterprise Analysts if
described utilizing our shopper’s enterprise terminology. That is notably related
when there’s not a Ubiquitous
Language shared between
technical and enterprise groups. Whereas the (enriched) information graph is the primary
product of the comprehension pipeline, it’s not the one precious one. Some
enrichments produced in the course of the pipeline, similar to mechanically generated
documentation in regards to the system, are precious on their very own. When supplied
on to customers, these enrichments can complement or fill gaps in present
techniques documentation, if one exists.

Determine 4: A comprehension pipeline in CodeConcise.

Neo4j, our graph database of alternative, holds the (enriched) Data Graph.
This DBMS options vector search capabilities, enabling us to combine the
Data Graph into the frontend software implementing RAG. This strategy
supplies the LLM with a a lot richer context by leveraging the graph’s construction,
permitting it to traverse neighboring nodes and entry LLM-generated explanations
at numerous ranges of abstraction. In different phrases, the retrieval element of RAG
pulls nodes related to the person’s immediate, whereas the LLM additional traverses the
graph to assemble extra info from their neighboring nodes. For example,
when searching for info related to a question about “how does authorization
work when viewing card particulars?” the index might solely present again outcomes that
explicitly cope with validating person roles, and the direct code that does so.
Nonetheless, with each behavioral and structural edges within the graph, we will additionally
embrace related info in known as strategies, the encompassing bundle of code,
and within the knowledge constructions which have been handed into the code when offering
context to the LLM, thus scary a greater reply. The next is an instance
of an enriched information graph for AWS Card
Demo,
the place blue and inexperienced nodes are the outputs of the enrichments executed within the
comprehension pipeline.

Determine 5: An (enriched) information graph for AWS Card Demo.

The relevance of the context supplied by additional traversing the graph
finally will depend on the standards used to assemble and enrich the graph within the
first place. There isn’t a one-size-fits-all answer for this; it’ll depend upon
the particular context, the insights one goals to extract from their code, and,
finally, on the rules and approaches that the event groups adopted
when establishing the answer’s codebase. For example, heavy use of
inheritance constructions would possibly require extra emphasis on INHERITS_FROM edges vs
COMPOSED_OF edges in a codebase that favors composition.

For additional particulars on the CodeConcise answer mannequin, and insights into the
progressive studying we had by way of the three iterations of the accelerator, we
will quickly be publishing one other article: Code comprehension experiments with
LLMs.

Within the subsequent sections, we delve deeper into particular modernization
challenges that, if solved utilizing GenAI, might considerably influence the price,
worth, and time for modernization – components that usually discourage us from making
the choice to modernize now. In some instances, we now have begun exploring internally
how GenAI would possibly tackle challenges we now have not but had the chance to
experiment with alongside our shoppers. The place that is the case, our writing is
extra speculative, and we now have highlighted these cases accordingly.

Reverse engineering: drawing out low-level necessities

When endeavor a legacy modernization journey and following a path
like Rewrite or Exchange, we now have realized that, to be able to draw a
complete checklist of necessities for our goal system, we have to
look at the supply code of the legacy system and carry out reverse
engineering. These will information your ahead engineering groups. Not all
these necessities will essentially be included into the goal
system, particularly for techniques developed over a few years, a few of which
might now not be related in at this time’s enterprise and market context.
Nonetheless, it’s essential to know present habits to make knowledgeable
choices about what to retain, discard, and introduce in your new
system.

The method of reverse engineering a legacy codebase may be time
consuming and requires experience from each technical and enterprise
individuals. Allow us to contemplate beneath a number of the actions we carry out to achieve
a complete low-level understanding of the necessities, together with
how GenAI may help improve the method.

Thoughtworks labored with the shopper talked about earlier to discover methods to
speed up the reverse engineering of a big legacy codebase written in COBOL/
IDMS. To realize this, we prolonged CodeConcise to assist the shopper’s tech
stack and developed a proof of idea (PoC) using the accelerator within the
method described above. Earlier than the PoC, reverse engineering 10,000 traces of code
sometimes took 6 weeks (2 FTEs working for 4 weeks, plus wait time and an SME
overview). On the finish of the PoC, we estimated that our answer might scale back this
by two-thirds, from 6 weeks to 2 weeks for a module. This interprets to a
potential saving of 240 FTE years for your complete mainframe modernization
program.

Excessive-level, summary rationalization of a system

We now have skilled that LLMs may help us perceive low-level
necessities extra shortly. The following query is whether or not they may
assist us with high-level necessities. At this stage, there’s a lot
info to absorb and it’s robust to digest all of it. To sort out this,
we create psychological fashions which function abstractions that present a
conceptual, manageable, and understandable view of the purposes we
are wanting into. Normally, these fashions exist solely in individuals’s heads.
Our strategy entails working carefully with specialists, each technical and
enterprise focussed, early on within the mission. We maintain workshops, similar to
Occasion
Storming
from Area-driven Design, to extract SMEs’ psychological fashions and retailer them
on digital boards for visibility, steady evolution, and
collaboration. These fashions include a site language understood by each
enterprise and technical individuals, fostering a shared understanding of a
complicated area amongst all workforce members. At a better stage of abstraction,
these fashions may additionally describe integrations with exterior techniques, which
may be both inner or exterior to the group.

It’s turning into evident that entry to, and availability of SMEs is
important for understanding complicated legacy techniques at an summary stage
in a cheap method. Lots of the constraints beforehand
highlighted are subsequently relevant to this modernization
problem.

Within the period of GenAI, particularly within the modernization area, we’re
seeing good outputs from LLMs when they’re prompted to clarify a small
subset of legacy code. Now, we wish to discover whether or not LLMs may be as
helpful in explaining a system at a better stage of abstraction.

Our accelerator, CodeConcise, builds upon Code as Knowledge strategies by
using the graph illustration of a legacy system codebase to
generate LLM-generated explanations of code and ideas at completely different
ranges of abstraction:

• Graph traversal technique: We leverage your complete codebase’s
  illustration as a graph and use traversal algorithms to complement the graph with
  LLM-generated explanations at numerous depths.
• Contextual information: Past processing the code and storing it within the
  graph, we’re exploring methods to course of any out there system documentation, as
  it typically supplies precious insights into enterprise terminology, processes, and
  guidelines, assuming it’s of excellent high quality. By connecting this contextual
  documentation to code nodes on the graph, our speculation is we will improve
  additional the context out there to LLMs throughout each upfront code rationalization and
  when retrieving info in response to person queries.

In the end, the purpose is to boost CodeConcise’s understanding of the
code with extra summary ideas, enabling its chatbot interface to
reply questions that sometimes require an SME, holding in thoughts that
such questions may not be straight answerable by inspecting the code
alone.

At Thoughtworks, we’re observing optimistic outcomes in each
traversing the graph and producing LLM explanations at numerous ranges
of code abstraction. We now have analyzed an open-source COBOL repository,
AWS Card
Demo,
and efficiently requested high-level questions similar to detailing the system
options and person interactions. On this event, the codebase included
documentation, which supplied further contextual information for the
LLM. This enabled the LLM to generate higher-quality solutions to our
questions. Moreover, our GenAI-powered workforce assistant, Haiven, has
demonstrated at a number of shoppers how contextual details about a
system can allow an LLM to supply solutions tailor-made to
the particular shopper context.

Discovering a functionality map of a system

One of many first issues we do when starting a modernization journey
is catalog present expertise, processes, and the individuals who assist
them. Inside this course of, we additionally outline the scope of what is going to be
modernized. By assembling and agreeing on these components, we will construct a
robust enterprise case for the change, develop the expertise and enterprise
roadmaps, and contemplate the organizational implications.
With out having this at hand, there is no such thing as a option to decide what wants
to be included, what the plan to attain is, the incremental steps to
take, and after we are completed.

Earlier than GenAI, our groups have been utilizing plenty of
strategies to construct this understanding, when it’s not already current.
These strategies vary from Occasion Storming and Course of Mapping by way of
to “following the information” by way of the system, and even focused code
critiques for notably complicated subdomains. By combining these
approaches, we will assemble a functionality map of our shoppers’
landscapes.
Whereas this may increasingly appear as if a considerable amount of guide effort, these can
be a number of the most precious actions because it not solely builds a plan for
the long run supply, however the considering and collaboration that goes into
making it ensures alignment of the concerned stakeholders, particularly
round what will be included or excluded from the modernization
scope. Additionally, we now have learnt that functionality maps are invaluable after we
take a capability-driven strategy to modernization. This helps modernize
the legacy system incrementally by step by step delivering capabilities in
the goal system, along with designing an structure the place
considerations are cleanly separated.

GenAI adjustments this image quite a bit.

One of the highly effective capabilities that GenAI brings is
the flexibility to summarize massive volumes of textual content and different media. We are able to
use this functionality throughout present documentation that could be current
concerning expertise or processes to extract out, if not the tip
information, then at the very least a place to begin for additional conversations.
There are a variety of strategies which can be being actively developed and
launched on this space. Particularly, we imagine that
GraphRAG which was just lately
launched by Microsoft might be used to extract a stage of information from
these paperwork by way of Graph Algorithm evaluation of the physique of
textual content.
We now have additionally been trialing GenAI excessive of the information graph
that we construct out of the legacy code as talked about earlier by asking what
key capabilities modules have after which clustering and abstracting these
by way of hierarchical summarization. This then serves as a map of
capabilities, expressed succinctly at each a really excessive stage and a
detailed stage, the place every functionality is linked to the supply code
modules the place it’s carried out. That is then used to scope and plan for
the modernization in a quicker method. The next is an instance of a
functionality map for a system, together with the supply code modules (small
grey nodes) they’re carried out in.

However, we now have learnt to not view this totally LLM-generated
functionality map as mutually unique from the standard strategies of
creating functionality maps described earlier. These conventional approaches
are precious not just for aligning stakeholders on the scope of
modernization, but additionally as a result of, when a functionality already exists, it
can be utilized to cluster the supply code based mostly on the capabilities
carried out. This strategy produces functionality maps that resonate higher
with SMEs by utilizing the group’s Ubiquitous language. Moreover,
evaluating each functionality maps is perhaps a precious train, absolutely one
we look ahead to experimenting with, as every would possibly supply insights the
different doesn’t.

Discovering unused / lifeless / duplicate code

One other a part of gathering info on your modernization efforts
is knowing inside your scope of labor, “what remains to be getting used at
all”, or “the place have we acquired a number of cases of the identical
functionality”.

At present this may be addressed fairly successfully by combining two
approaches: static and dynamic evaluation. Static evaluation can discover unused
methodology calls and statements inside sure scopes of interrogation, for
occasion, discovering unused strategies in a Java class, or discovering unreachable
paragraphs in COBOL. Nonetheless, it’s unable to find out whether or not entire
API endpoints or batch jobs are used or not.

That is the place we use dynamic evaluation which leverages system
observability and different runtime info to find out if these
features are nonetheless in use, or may be dropped from our modernization
backlog.

When seeking to discover duplicate technical capabilities, static
evaluation is essentially the most generally used software as it will possibly do chunk-by-chunk textual content
similarity checks. Nonetheless, there are main shortcomings when utilized to
even a modest expertise property: we will solely discover code similarities in
the identical language.
We speculate that by leveraging the results of {our capability}
extraction strategy, we will use these expertise agnostic descriptions
of what massive and small abstractions of the code are doing to carry out an
estate-wide evaluation of duplication, which is able to take our future
structure and roadmap planning to the following stage.

In relation to unused code nevertheless, we see little or no use in
making use of GenAI to the issue. Static evaluation instruments within the trade for
discovering lifeless code are very mature, leverage the structured nature of
code and are already at builders’ fingertips, like IntelliJ or Sonar.
Dynamic evaluation from APM instruments is so highly effective there’s little that instruments
like GenAI can add to the extraction of knowledge itself.

Then again, these two complicated approaches can yield an enormous
quantity of knowledge to know, interrogate and derive perception from. This
might be one space the place GenAI might present a minor acceleration
for discovery of little used code and expertise.
Much like having GenAI discuss with massive reams of product documentation
or specs, we will leverage its information of the static and
dynamic instruments to assist us use them in the correct method as an example by
suggesting potential queries that may be run over observability stacks.
NewRelic, as an example, claims to have built-in LLMs in to its options to
speed up onboarding and error decision; this might be turned to a
modernization benefit too.

Idiomatic translation of tech paradigm

Translation from one programming language to a different isn’t one thing new. Many of the instruments that do that have
utilized static evaluation strategies – utilizing Summary Syntax Bushes (ASTs) as intermediaries.

Though these strategies and instruments have existed for a very long time, outcomes are sometimes poor when judged by way of
the lens of “would somebody have written it like this if that they had began authoring it at this time”.

Usually the produced code suffers from:

Assuming a corporation embarks on this journey, it’ll quickly face prolonged testing and verification
cycles to make sure the generated code behaves precisely the identical method as earlier than. This turns into much more difficult
when little to no security internet was in place initially.

Regardless of all of the drawbacks, code conversion approaches proceed to be an possibility that draws some organizations
due to their attract as doubtlessly the bottom value/ effort answer for leapfrogging from one tech paradigm
to the opposite.

We now have additionally been interested by this and exploring how GenAI may help enhance the code produced/ generated. It
can not help all of these points, however perhaps it will possibly assist alleviate at the very least the primary three or 4 of them.

From an strategy perspective, we are attempting to use the rules of
Refactoring
to this – primarily
determine a method we will safely and incrementally make the soar from one tech paradigm to a different. This strategy
has already seen some success – two examples that come to thoughts:

Conclusion

In the present day’s panorama has quite a few alternatives to leverage GenAI to
obtain outcomes that had been beforehand out of attain. Within the software program
trade, GenAI is already enjoying a big position in serving to individuals
throughout numerous roles full their duties extra effectively, and this
influence is anticipated to develop. For example, GenAI has produced promising
leads to helping technical engineers with writing code.

Over the previous a long time, our trade has advanced considerably, creating patterns, greatest practices, and
methodologies that information us in constructing fashionable software program. Nonetheless, one of many largest challenges we now face is
updating the huge quantity of code that helps key operations each day. These techniques are sometimes massive and sophisticated,
with a number of layers and patches constructed over time, making habits troublesome to vary. Moreover, there are
typically just a few specialists who totally perceive the intricate particulars of how these techniques are carried out and
function. For these causes, we use an evolutionary strategy to legacy displacement, lowering the dangers concerned
in modernizing these techniques and producing worth early. Regardless of this, the price/time/worth equation for
modernizing massive techniques is usually prohibitive. On this article, we mentioned methods GenAI may be harnessed to
flip this example round. We are going to proceed experimenting with making use of GenAI to those modernization challenges
and share our insights by way of this text, which we are going to maintain updated. It will embrace sharing what has
labored, what we imagine GenAI might doubtlessly resolve, and what, nevertheless, has not succeeded. Moreover, we
will lengthen our accelerator, CodeConcise, with the goal of additional innovating inside the modernization course of to
drive higher worth for our shoppers.

Hopefully, this text highlights the nice potential of harnessing
this new expertise, GenAI, to deal with a number of the challenges posed by
legacy techniques within the trade. Whereas there is no such thing as a one-size-fits-all
answer to those challenges – every context has its personal distinctive nuances –
there are sometimes similarities that may information our efforts. We additionally hope this
article conjures up others within the trade to additional develop experiments
with “GenAI for Modernization” and share their insights with the broader
group.