Building GitHub Search Filters in Go

RileyCoder Novice 11h ago 56 views 15 likes 2 min read

Parsing search strings like is:open label:bug author:me usually feels like a descent into regex madness. Most developers start with strings.Split or strings.HasPrefix, but that falls apart the moment you encounter quoted strings with spaces (e.g., label:"help wanted") or comparison operators like created:>2024-01-01.

Building GitHub Search Filters in Go

The cleanest way to handle this is to treat it as a formal parsing problem rather than string munging. I used a library called participle, which allows you to define a grammar using plain Go structs. No separate grammar files or complex code generation—the struct is the specification.

The Problem with Manual Parsing

Avoid the "I'll clean it up later" approach of chaining if/else blocks. This snippet is a classic example of a fragile implementation:

Building GitHub Search Filters in Go

// Fragile manual parsing
parts := strings.Fields(query)
for _, p := range parts {
 if strings.HasPrefix(p, "label:") {
     labels = append(labels, strings.TrimPrefix(p, "label:"))
 } else if strings.HasPrefix(p, "is:") {
     state = strings.TrimPrefix(p, "is:")
 }
 // This breaks immediately with quoted strings or operators
}

Implementing a Typed AST

By using participle, you can define an Abstract Syntax Tree (AST) that maps directly to your search language. I've implemented a "GitHub minimal" version that supports qualifiers, comparisons, free text, and negation (the - prefix).

Building GitHub Search Filters in Go

The entire grammar is handled by three focused structs:

// Query represents the full search string as a list of ANDed terms
type Query struct {
    Terms []*Term `parser:"@@*"`
}

// Term handles either a key:value qualifier or a raw search word,
// including optional negation.
type Term struct {
    Negated    bool       `parser:"@Dash?"`
    Qualifier  *Qualifier `parser:"( @@"`
    Text       *Value     `parser:"| @(String | Ident | Number) )"`
}

![Building GitHub Search Filters in Go](/uploads/articles/b069f1c29b5013af.jpg)

// Qualifier maps to "key:value" pairs like is:open
type Qualifier struct {
    Key   string `parser:"@Ident"`
    Colon string `parser:"@Colon"`
    Value string `parser:"@Ident"`
}

Real-World Execution

Once the parser converts the string into a typed syntax tree, the next step is compiling that tree into safe, parameterized SQL to avoid injection.

When running this against a local dataset, the output is precise. A query like is:open label:bug correctly filters the database and returns only the matching issues:

$ make run q='is:open label:bug'
query: is:open label:bug
6 issue(s)

# STATE AUTHOR CREATED COMMENTS LABELS TITLE
1 open octocat 2024-02-10 12 bug,urgent Login button unresponsive on mobile
6 open bob 2024-04-18 15 bug,p0,urgent Memory leak in worker pool

This approach transforms a messy string-processing task into a structured AI workflow for data retrieval. If you're building a custom filtering system, moving from regex to a struct-based parser is a massive productivity gain.

AI ProgrammingAI Codingbeginnerswebdevprogramming

All Replies (3)

S
Sam46 Advanced 11h ago
I once spent three days fighting a regex for this. Just use a parser.
0 Reply
J
JulesCrafter Novice 11h ago
Tried a custom lexer for this once. Way cleaner than nesting a dozen if-statements.
0 Reply
G
GhostGeek Expert 11h ago
Worth looking into text/scanner for this; saves a lot of manual slicing.
0 Reply

Write a Reply

Markdown supported