[Geany] Error transformer

[Laszlo Nagy]

   Hi All,

I was writting a compiler for a new language, and I was having problems 
with Geany. Whenever I tried to run "make" (e.g. press F8 in geany) 
there were three cases:

* no error
* error in the compiler program -> error message was a Python traceback
* error in the source file being compiled by the compiler -> error 
message was a gnu formatted error message.

I asked about this problem here before: what kind of regular expression 
should I use so that Geany finds both errors? The answer was that it is 
almost impossible, because Geany can only handle a single regular 
expression.

Today I came up with another solution. It is a tiny wrapper program that 
can run any other command. It monitors the data that is flowing through 
stderr and stdout. It can run several different regular expressions on 
the output, and convert them into standardized format.

It is trivial to change this program so that you parse the compiler 
output from program code (instead of a regular expression). It allows 
you to parse the output of almost anything, including cases when the 
information is spread in multiple lines.

If anyone is interested, the "program" is attached. I'm not sure if it 
worth adding it to the Geany wiki.

Best,

    Laszlo

-------------- next part --------------
#!/usr/bin/env python

#
# Transform compiler messages into gnu format so that Geany can locate
# the files automatically. Inside Geany, use this error regex:
#
# ^([^:]+?):([0-9]+):.+
#
# You can change the "PATTERNS" constant below, to your needs.
#
#
import os
import sys
import re
import functools
from subprocess import Popen, PIPE
from threading  import Thread
try:
    from Queue import Queue, Empty
except ImportError:
    from queue import Queue, Empty  # python 3.x
ON_POSIX = 'posix' in sys.builtin_module_names


#
# Add new languages here with their own regular expressions here.
#
# "pat" is the pattern for parsing. All other fields are regex group
# numbers. You can also use constant strings instead of numbers (e.g.
# when a field is not available)
#
PATTERNS = {
    "python": {
        'pat': re.compile(
          r"""\s*File\s+"([^"]+)",\s+line\s+(\d+),\s+in\s+([^\s].*)"""),
        'fpath':  0, 'lineno': 1, 'level': 'E', 'message': 'Python error',
    }
}


def enqueue_output(out, queue):
    for line in iter(out.readline, b''):
        queue.put(line)
    out.close()


def transform(args, shell, outproc, errproc):
    """Process a command's output.

    @param args: Arguments to start the program.
    @param shell: Set True to execute it with the shell.
    @param outproc: Callback to process a line of stderr.
    @param errproc: Callback to process a line of stderr.
    @return: Exit code of the application.
    """
    proc = Popen(args, shell=shell, stdout=PIPE, stderr=PIPE,
        bufsize=1, close_fds=ON_POSIX)
    qo = Queue()
    to = Thread(target=enqueue_output, args=(proc.stdout, qo))
    to.daemon = True
    to.start()
    qe = Queue()
    te = Thread(target=enqueue_output, args=(proc.stderr, qe))
    te.daemon = True
    te.start()

    def proc_output(final=False):
        while True:
            try:
                line = qo.get_nowait()
                outproc(line)
            except Empty:
                pass

            try:
                line = qe.get_nowait()
                errproc(line)
            except Empty:
                pass

            if not final:
                break

            if qo.empty() and qe.empty():
                break

    while True:
        terminated = proc.poll() is not None
        if terminated:
            break
        proc_output()
    proc_output(True)

    return proc.returncode


def _get_pitem(res, props, propname):
    if propname in props:
        ret = props[propname]
        if isinstance(ret, int):
            return res[ret]
        else:
            return ret
    else:
        return ""


def procline(fout, s):
    global PATTERNS
    fout.write(s)
    fout.flush()
    for key, props in PATTERNS.iteritems():
        sre = props["pat"].match(s)
        if sre:
            res = sre.groups()
            fout.write(u'"%s":%s:%s:%s\n' % (
                _get_pitem(res, props, "fpath"),
                _get_pitem(res, props, "lineno"),
                _get_pitem(res, props, "level"),
                _get_pitem(res, props, "message"),
            ))
            fout.flush()

raise SystemExit(transform(sys.argv[1:], False,
    functools.partial(procline, sys.stdout),
    functools.partial(procline, sys.stderr),
))