Sierra Flare

My journey with Flare-on 12.

The Flare

In September of last year, @crazyman told me about an ongoing reverse-engineering CTF. That was the first time I learned about Flare-On (and also that it used to be part of Mandiant and is now owned by Google). With the experience from last year, I decided to dedicate myself more to this year’s competition and see if I could achieve a good result. In the end, I completed all the challenges in 4 days, 21 hours, and 33 minutes.

This year’s problems seemed a bit easier, but they were still very interesting. Personally, I spent most of my time on Challenge 5 and Challenge 9. The ninth one, which is also the final challenge, wasn’t actually that complex in my opinion. However, the time required can vary greatly depending on a participant’s attention to detail and level of familiarity. Every small mistake, whether it’s a wrong decision or a script error, can appreciably increase the total time spent.

9 - 10000

An (semi-)auto-rev challenge.

As the name suggests, the 1GB challenge file contains 10,000 resource elements.
While the main logic is pretty straightforward, it only does four things without any obfuscation:

• open the license.bin, which should be an array of 10,000 [2-byte index][32-byte key] pairs.
• Load No.n resource according to the index, and let the _Z5checkPh function in loaded resource to check the 32-byte key.
• If correct, for each loaded resource indexed x at this round, add target[x] with round counter (resources can depend on each other).
• Compare the final target array with expected array. If equal, use SHA256 of license.bin as AES key to decrypt the flag.

So we have two tasks:

• Reverse all 10,000 _Z5checkPh functions to get the expected 32-byte keys.
• Recover the correct order of resource sequence to make the final target array equal to expected array.

At a glance, we might pick whichever task we want to do first.
However, after a closer look at some random resources, we find that each sub-check function is referencing a global variable which is actually the target[library index].

Note: The subcheck functions use a user-defined calling convention, and IDA will fail to guess the arguments, thus it will take a long time to decompile the _Z5checkPh function.
We can automatically fix all subcheck function prototypes by the script:

import idautils, ida_typeinf  # IDA 7.5+
idati = ida_typeinf.get_idati()
for ea, name in idautils.Names():
    if name.startswith('_Z21f'):
        decl = "void __usercall {}(_BYTE *ptr@<rcx>);".format(name)
        tif = ida_typeinf.tinfo_t()
        ida_typeinf.parse_decl(tif, idati, decl, ida_typeinf.PT_SIL)
        ida_typeinf.apply_tinfo(ea, tif, ida_typeinf.TINFO_DEFINITE)

This means we have to first recover the correct order of resource sequence to make the final target array equal to expected array, and then handle those _Z5checkPh functions in the resources.

Recover the correct order of resource (calling) sequence

Here I describe the task from the reversed logic and asked ChatGPT-5 to solve this PPC-like problem.

SEQ is a permutation of integers 0..9999 (length 10000).
SRC is a given **DAG** with 10000 nodes (1-based index).
SRC[i] contains between 1 and 10,000 distinct integers drawn from 1..10,000
CHANGES = reachability(SRC)

Goal: Recover the unique permutation SEQ that satisfies:
RESULT = [0]*10000
for i in range(10000):
    for j in CHANGES[SEQ[i]]:
        RESULT[j] += i

assert RESULT == a fixed known array of length 10000.

The SRC array can be easily recovered with dependency relation exported from pefile.

And GPT-5 immediately told us how to solve it with Möbius Inversion:

def solve_seq(GOALS, CHANGES):  # O(M)
    n = len(GOALS)
    Anc = [[] for _ in range(n)]
    for k, reach in enumerate(CHANGES):
        for j in reach:
            Anc[j].append(k)
    order = list(range(n))
    order.sort(key=lambda j: len(Anc[j]))
    y = [None] * n
    for j in order:
        s = 0
        for k in Anc[j]:
            if k == j:
                continue
            val = y[k]
            s += val
        yj = GOALS[j] - s
        y[j] = yj
    SEQ = [0] * n
    for k, i in enumerate(y):
        SEQ[i] = k
    return SEQ

Now that we have the correct resource calling sequence, we can calculate the state of the global value (target array) before calling each _Z5checkPh function, it’s time to reverse all 10,000 of them.

It’s just 10,000 todos

Each resource is a PE file that contains a _Z5checkPh function and dozens of subcheck functions. But in general (as with a beginner-friendly challenge), there are only four types of checks:

• S_BOX transformation in subcheck functions.
• byte swapping in subcheck functions.
• A odd-mod-2^n exponent calculation in subcheck functions.
• A series of matrix operations in _Z5checkPh function.

And all constants involved have the same relative offset with function start.
The only difficulty is the series of matrix operations; but again, with the help of GPT-5 we knew how to invert it:

def invert_main(_const):
    mod_p = int(_const[0])
    e_pow = int(_const[1])
    CONST_C_16 = [int(x) for x in _const[2:10]] + [int(x) for x in _const[10:18]]
    T_vals = [int(x) % mod_p for x in _const[18:34]]
    ord_T = Matrix(GF(mod_p), 4, 4, T_vals).multiplicative_order()
    d = pow(e_pow, -1, int(ord_T))
    F = GF(mod_p)
    A = Matrix(F, 4, 4, [F(int(x)) for x in T_vals])
    R = A ^ Integer(d)
    A = [int(a) for a in R.list()]
    mask64 = (1 << 64) - 1
    a4_u64 = [None] * 4
    for i in range(16):
        col = i % 4
        Ai = A[i] % mod_p
        Ci = CONST_C_16[i] & mask64
        guess = (Ai ^ Ci) & mask64
        if a4_u64[col] is None:
            a4_u64[col] = guess
    out = []
    for v in a4_u64:
        b = int(v).to_bytes(8, "little")
        out.extend(b[i] for i in range(8))
    return out

Cat flag

With all the extracted constants and dependency relations, we calculated the calling sequence and reversed all 10,000 PE resources to obtain the expected 32-byte keys.
We then constructed the correct license.bin file:

And decoded the flag:

8 - FlareAuthenticator

This is a large Qt application with MBA obfuscation, but we were able to recover the correct passcode with dynamic debugging and a little observation.
(I saw some amazing writeups that use symbolic execution to solve it directly and even use symbolic composition to recreate the const generation logic, wow!)

After breaking the program, hit the press check button and after a few run-tail-return commands, we could see the check logic at 0x140021E29.
We then set a memory breakpoint to observe how the checked value was generated:

keys = [unknown_value_between(0-9) for _ in range(25)]
for index in range(1, 26):
    key = keys[index]
    current += bucket_a[index] * bucket_b[index][key]
    current &= 0xffffffffffffffff

current must equal 0xBC42D5779FEC401 at the end (64-bit check)

Where bucket_a[index] = hash_x(index)
bucket_b[index][i] = hash_x((index << 8) | 0x30 | keys[i])

The algorithm seems clear enough and the unknown hash_x function has maximum 10 + 10 * 25 = 260 possible inputs. So I decided to extract all results of hash_x and see if we could solve it directly (without a deeper understanding of hash_x).

I added two breakpoints in IDA to dump the values of bucket_a and bucket_b:

0x140016772  # 0x140016B00
print(hex(ida_dbg.get_reg_val("rax")), hex(ida_dbg.get_reg_val("rcx")))

Before feeding all constraints to Z3, I asked GPT to see if it could have any suggestions and it provided a plan using heuristic DFS with pruning and can find the solution in a second:

v = [[bucket_a[i] * bucket_b[i][k] for k in range(10)] for i in range(25)]
T = 0xBC42D5779FEC401 
opts = [sorted(((v[i][k], k) for k in range(10)), key=lambda x: x[0]) for i in range(25)]
order = sorted(range(25), key=lambda i: (opts[i][-1][0] - opts[i][0][0]), reverse=True)
mins = [0] * 26
maxs = [0] * 26
for p in range(24, -1, -1):
    i = order[p]
    mins[p] = mins[p + 1] + opts[i][0][0]   
    maxs[p] = maxs[p + 1] + opts[i][-1][0] 

solution = [None] * 25
def dfs(p: int, acc: int):
    if acc + mins[p] > T or acc + maxs[p] < T:
        return
    if p == 25:
        if acc == T:
            print(solution)
            exit(0)
    i = order[p]
    target_here = T - (acc + mins[p + 1])
    cand_ids = list(range(10))
    cand_ids.sort(key=lambda j: abs(opts[i][j][0] - target_here))
    for j in cand_ids:
        val, k = opts[i][j]
        solution[i] = k
        dfs(p + 1, acc + val)
    solution[i] = None


dfs(0, 0) v = [[bucket_a[i] * bucket_b[i][k] for k in range(10)] for i in range(25)]
T = 0xBC42D5779FEC401 
opts = [sorted(((v[i][k], k) for k in range(10)), key=lambda x: x[0]) for i in range(25)]
order = sorted(range(25), key=lambda i: (opts[i][-1][0] - opts[i][0][0]), reverse=True)
mins = [0] * 26
maxs = [0] * 26
for p in range(24, -1, -1):
    i = order[p]
    mins[p] = mins[p + 1] + opts[i][0][0]   
    maxs[p] = maxs[p + 1] + opts[i][-1][0] 

solution = [None] * 25
def dfs(p: int, acc: int):
    if acc + mins[p] > T or acc + maxs[p] < T:
        return
    if p == 25:
        if acc == T:
            print(solution)
            exit(0)
    i = order[p]
    target_here = T - (acc + mins[p + 1])
    cand_ids = list(range(10))
    cand_ids.sort(key=lambda j: abs(opts[i][j][0] - target_here))
    for j in cand_ids:
        val, k = opts[i][j]
        solution[i] = k
        dfs(p + 1, acc + val)
    solution[i] = None


dfs(0, 0)

Now we can let the program derive the key and decrypt the flag (without touching the MBA obfuscation).

7 - The Boss Needs Help

Here is where things started to get complex, we got a 4MB PE binary named hopeanddreams.exe, hmm, hopes.

Basically this is a RAT program and we also have the encrypted traffic between the RAT and its C2 server. So our task is to reverse the communication protocol (especially for key exchange) and decrypt the traffic data.

At 0x140081300 we can find unobfuscated logic of AES key = XOR(sha256(A||fmt(time())), sha256(C)), but expect there, the MBA obf (Mixed Boolean-Arithmetic Obfuscation) is everywhere:

But if we look closely, we will notice that MBA are just red herrings here, which keep reading and writing some global variables but are NEVER involved in the final calculation of enc/dec logic:

So again, I asked GPT-5 to write a taint analysis script to track the data flow starting from mov reg32, cs:{four global vars} and NOPed all tainted instructions.
This approach was based on the assumption that the final encryption/decryption logic does not depend on those four global variables or any MBA-tainted data.

After NOPing hundreds of thousands of instructions, I finally obtained clean logic without any (MBA) obfuscation, and now we are able to decode the encrypted strings like “https://www.youtube.com/watch?v=6O3MO2y30fU&…” or “cake is a lie“ (I did not expect to see my this year nickname here XD).

Of course, we were able to decrypt some initial traffic data, including the handshake:

cipher = bytes.fromhex('e4b8058f06f7061e8f0f8ed15d23865ba2427b23a695d9b27bc308a26d')
r = []
for i in range(len(cipher)):
    x = INV_S[cipher[i]] - i - 1
    if x < 0:
        x += 256
    x ^= 0x5a
    r.append(x)
print(bytes(r))  # 2025082006TheBoss@THUNDERNODE

cipher = bytes.fromhex('...')
c = "TheBoss@THUNDERNODE"
for i in range(len(cipher)):
    x = INV_S[cipher[i]]
    x = x - i - 1
    if x < 0:
        x += 256
    x ^= ord(c[i % len(c)])
    r.append(x)  # peanut@...

We then set the AES key to XOR(SHA256(peanut06), SHA256(TheBoss@THUNDERNODE)).
Repeating this for three more key-update rounds (XOR(SHA256([newkey]06), SHA256(TheBoss@THUNDERNODE))) allowed us to decrypt the entire traffic.

The following traffic transferred an encrypted zip file:

And at one of the last packages, we can find the password list:

Email: BornToRun!75
Bank: TheRiver##1980
ComputerLogin: TheBossMan
Other: TheBigM@n1942!

We now had everything to help The Boss.

6 Chain of Demands

Again a pyinstaller packed binary, decompress and drop it to https://pylingual.io/ to get the readable source code.

This looks like a crypto challenge, which GPT is good at, and it turns out GPT can cook it without a single interaction:

...
We calculated the modulus m using a formula and simulated LCG to find eight 256-bit prime numbers to construct the RSA modulus. This allowed us to successfully decrypt the message. The final decrypted message contained:

1: "What is your email address actually?"

2: "It's W3b3_i5_Gr8@flare-on.com"

The key identifier extracted is either "W3b3_i5_Gr8" or the entire email address. These are the possible identifiers.

5 ntfsm

A maze challenge, simply export the .S file from IDA, build graph with regex, and find the longest path in the DAG.

import re
import networkx as nx
from pwn import u32

f=open("ntfsm.s","r")
x=f.read()
f.close()

f = open("ntfsm.exe", "rb")
f.seek(0xC67BB8)
raw_table = f.read()
table = []
for i in range(0, (0x1629C + 2) * 4, 4):
    table.append(0x140000000 + u32(raw_table[i: i + 4]))
f.close()


kj=re.findall(r"rax, [0-9A-F]+h\n.*jl *short (loc_[0-9A-F]+)\n.*?\n.*?\n.*cmp     byte ptr \[rsp\+[0-9A-F]+h\], ..h ; '(.)'\n.*?jz      short (loc_[0-9A-F]+)\n(.*?\n.*?\n)(.*?\n.*?\n)", x)  

fj=re.findall(r"(loc_140......):.*?\n.*mov     qword ptr \[rsp+.*?, ([0-9A-F]*).*?\n", x)
real_to_table={}
for i in fj:
    k,v = i
    real_to_table[k]=v

G = nx.DiGraph()

cnt=0
for i in kj:
    raw_from, edge, raw_to, ext_a, ext_b = i
    raw_from = int(raw_from[4:], 16) - 0x11
    real_from = 'loc_' + hex(raw_from)[2:].upper()
    real_to = int(real_to_table[raw_to], 16)
    assert real_to <= 0x1629C
    real_to = 'loc_' + hex(table[real_to])[2:].upper()
    G.add_node(real_from)
    G.add_node(real_to)
    G.add_edge(real_from, real_to, label=edge)
    for ext in [ext_a, ext_b]:
        if 'cmp' in ext and 'jz' in ext:
            try:
                ext_edge, raw_ext_to = re.findall(r"cmp     byte ptr \[rsp\+[0-9A-F]+h\], ..h ; '(.)'\n.*?jz      short (loc_[0-9A-F]+)\n", ext)[0]
                ext_real_to = int(real_to_table[raw_ext_to], 16)
                assert ext_real_to <= 0x1629C
                ext_real_to = 'loc_' + hex(table[ext_real_to])[2:].upper()
                G.add_node(ext_real_to)
                G.add_edge(real_from, ext_real_to, label=ext_edge)
            except Exception as e:
                print(e)



def all_longest_paths_dag(G: nx.DiGraph, weight: str | None = None, label: str = "label"):
    if not nx.is_directed_acyclic_graph(G):
        raise ValueError("NOT DAG")

    topo = list(nx.topological_sort(G))
    dist = {v: 0 for v in topo}
    paths = {v: [[v]] for v in topo} 
    def edge_len(u, v):
        if weight is None:
            return 1
        return float(G[u][v].get(weight, 0))
    for v in topo:
        for u in G.predecessors(v):
            cand = dist[u] + edge_len(u, v)
            if cand > dist[v]:
                dist[v] = cand
                paths[v] = [p + [v] for p in paths[u]]
            elif cand == dist[v]:
                paths[v].extend(p + [v] for p in paths[u])

    max_len = max(dist.values())
    longest_paths_nodes = [p for v in topo for p in paths[v] if dist[v] == max_len]
    longest_paths_edges = []
    for nodes in longest_paths_nodes:
        edges = [(u, v, G[u][v].get(label)) for u, v in zip(nodes[:-1], nodes[1:])]
        longest_paths_edges.append((nodes, edges))

    return max_len, longest_paths_edges

max_len, longest = all_longest_paths_dag(G)

print("max len:", max_len)
print("count:", len(longest))
for nodes, edges in longest:
    print("nodes:", nodes)
    print("edges:", edges)
    print(''.join([i[2] for i in edges]))

4 Unholy Dragon

Rename the file to UnholyDragon-0.exe and run it until it stops generating new files.
Apply the diff back, and the flag pops out.

3 pretty devilish file

A challenge to deal with some PDF magic, throw it to GPT and retrieve the flag.

2 project chimera & 1 DrillBabyDrill

You didn’t come here for these two, right?

End

It’s time to plan a trip to Seattle and treat myself to a Black Forest cake.