update to python 3.8+

[a16bitsysop]

• update util.py for python 3.8+
• fixes Test failure on Python 3.11 #4

[a16bitsysop]

I have been trying to update to work with python 3.10 on alpine linux, but with the following patch I still get:

====================================================== test session starts =======================================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
rootdir: /home/devops/aports/testing/py3-pypytools/src/pypytools-0.6.2
plugins: mock-3.5.1, cov-2.12.1, forked-1.3.0, xdist-1.33.0
collected 101 items

pypytools/gc/testing/test_custom.py .....s.                                                                                [  6%]
pypytools/gc/testing/test_fakegc.py ...                                                                                    [  9%]
pypytools/gc/testing/test_multihook.py ........                                                                            [ 17%]
pypytools/gc/testing/test_uniform.py ..........                                                                            [ 27%]
pypytools/pypylog/testing/test_model.py .....                                                                              [ 32%]
pypytools/pypylog/testing/test_parse.py .F....F                                                                            [ 39%]
pypytools/testing/test_cast.py .                                                                                           [ 40%]
pypytools/testing/test_codegen.py ......................                                                                   [ 62%]
pypytools/testing/test_fakecython.py ..                                                                                    [ 64%]
pypytools/testing/test_micronumpy.py ...........................                                                           [ 91%]
pypytools/testing/test_unroll.py ........                                                                                  [ 99%]
pypytools/testing/test_util.py .                                                                                           [100%]

============================================================ FAILURES ============================================================
__________________________________________________ TestFlatParser.test_mismatch __________________________________________________

self = <pypytools.pypylog.testing.test_parse.TestFlatParser object at 0xffffa78f9420>

    def test_mismatch(self):
        # raise an error for now, but I think there might be real cases in
        # which this happens
        log = """
        [123] {foo
        [789] {bar
        [456] foo}
        [0ab] bar}
        """
>       pytest.raises(parse.ParseError, self.parse(log))

pypytools/pypylog/testing/test_parse.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pypytools/pypylog/testing/test_parse.py:13: in parse
    return parse.flat(f, log)
pypytools/pypylog/parse.py:42: in from_file
    return parse_file(fname)
pypytools/pypylog/parse.py:33: in parse_file
    p.feed(f)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pypytools.pypylog.parse.FlatParser object at 0xffffa78fa470>, f = <_io.StringIO object at 0xffffa78eacb0>

    def feed(self, f):
        zero_ts = None
        stack = []
        for line in f:
            if line == '\n':
                continue
            kind, tsid, ts, name = self.parse_line(line)
            if zero_ts is None:
                zero_ts = ts
            if kind == 'start':
                stack.append(Section(tsid, name, start=ts-zero_ts))
            elif kind == 'stop':
                if name != stack[-1].name:
                    msg = "End section does not match start: expected %s, got %s"
>                   raise ParseError(msg % (stack[-1].name, name))
E                   pypytools.pypylog.parse.ParseError: End section does not match start: expected bar, got foo

pypytools/pypylog/parse.py:77: ParseError
______________________________________________________ test_parse_frequency ______________________________________________________

    def test_parse_frequency():
        pf = parse.parse_frequency
        assert pf('40') == 40
        assert pf('40hz') == 40
        assert pf('40Hz') == 40
        assert pf('40 Hz') == 40
        assert pf('40 Hz ') == 40
        assert pf('40 KHz') == 40e3
        assert pf('40 MHz') == 40e6
        assert pf('40 GHz') == 40e9
>       pytest.raises(ValueError, "Invalid frequency")
E       TypeError: 'Invalid frequency' object (type: <class 'str'>) must be callable

pypytools/pypylog/testing/test_parse.py:127: TypeError
==================================================== short test summary info =====================================================
FAILED pypytools/pypylog/testing/test_parse.py::TestFlatParser::test_mismatch - pypytools.pypylog.parse.ParseError: End section...
FAILED pypytools/pypylog/testing/test_parse.py::test_parse_frequency - TypeError: 'Invalid frequency' object (type: <class 'str...
============================================ 2 failed, 98 passed, 1 skipped in 2.49s =============================================

[antocuni]

Note that I'm occasionally still using pypytools on py27, so I don't want a PR which removes python2 support.
That said, PRs to add support for more py3 versions are very welcome

[a16bitsysop]

Okay, will add back python2 stuff. Can you tell what is happening in the test failures?

[a16bitsysop]

I think this is now python2 and 3 compatible, but i still get:

============================================================ FAILURES ============================================================
__________________________________________________ TestFlatParser.test_mismatch __________________________________________________

self = <pypytools.pypylog.testing.test_parse.TestFlatParser object at 0xffffa6e81990>

    def test_mismatch(self):
        # raise an error for now, but I think there might be real cases in
        # which this happens
        log = """
        [123] {foo
        [789] {bar
        [456] foo}
        [0ab] bar}
        """
>       pytest.raises(parse.ParseError(self.parse(log)))

pypytools/pypylog/testing/test_parse.py:43:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pypytools/pypylog/testing/test_parse.py:20: in parse
    return parse.flat(f, log)
pypytools/pypylog/parse.py:45: in from_file
    return parse_file(fname)
pypytools/pypylog/parse.py:34: in parse_file
    p.feed(f)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pypytools.pypylog.parse.FlatParser object at 0xffffa6e81b70>, f = <_io.StringIO object at 0xffffb36be950>

    def feed(self, f):
        zero_ts = None
        stack = []
        for line in f:
            if line == '\n':
                continue
            kind, tsid, ts, name = self.parse_line(line)
            if zero_ts is None:
                zero_ts = ts
            if kind == 'start':
                stack.append(Section(tsid, name, start=ts-zero_ts))
            elif kind == 'stop':
                if name != stack[-1].name:
                    msg = "End section does not match start: expected %s, got %s"
>                   raise ParseError(msg % (stack[-1].name, name))
E                   pypytools.pypylog.parse.ParseError: End section does not match start: expected bar, got foo

pypytools/pypylog/parse.py:80: ParseError
______________________________________________________ test_parse_frequency ______________________________________________________

    def test_parse_frequency():
        pf = parse.parse_frequency
        assert pf('40') == 40
        assert pf('40hz') == 40
        assert pf('40Hz') == 40
        assert pf('40 Hz') == 40
        assert pf('40 Hz ') == 40
        assert pf('40 KHz') == 40e3
        assert pf('40 MHz') == 40e6
        assert pf('40 GHz') == 40e9
>       pytest.raises(ValueError(pf('')))

pypytools/pypylog/testing/test_parse.py:134:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

s = ''

    def parse_frequency(s):
        """
        Parse an human-readable string and return the frequency expressed in
        hertz.  It supports different units such as Hz, hz, MHz, GHz, etc.
        """
        s = s.lower().strip()
        if s.endswith('hz'):
            s = s[:-2]
        if not s:
>           raise ValueError
E           ValueError

pypytools/pypylog/parse.py:149: ValueError
==================================================== short test summary info =====================================================
FAILED pypytools/pypylog/testing/test_parse.py::TestFlatParser::test_mismatch - pypytools.pypylog.parse.ParseError: End section...
FAILED pypytools/pypylog/testing/test_parse.py::test_parse_frequency - ValueError
============================================ 2 failed, 98 passed, 1 skipped in 2.55s =============================================

[a16bitsysop]

updated after ran tests under debian python2.7 via docker, so now runs on 2.7 but same two xfails

[ooliver1]

Note that I'm occasionally still using pypytools on py27, so I don't want a PR which removes python2 support. That said, PRs to add support for more py3 versions are very welcome

2.7 is 12 years old, dont you think that current versions should be prioritized over incredibly old EOL versions

[antocuni]

2.7 is 12 years old, dont you think that current versions should be prioritized over incredibly old EOL versions

2.7 is still used in many places, including some which I care about. So no, I'm not going to remove 2.7 support but as I said I'm more than happy to support more py3.x versions

[a16bitsysop]

This does not remove 2.7 anymore

[webknjaz]

@a16bitsysop it'd be useful to integrate GHA so that the test run results are visible transparently.

[a16bitsysop]

thanks for the review, added all the suggestions

[sertonix]

Could you also add this to fix 3.11 and above? That would fix #4

    if PY3 and PY3M >= 11:
        args.insert(12, co.co_qualname)
        args.insert(15, co.co_exceptiontable)

[a16bitsysop]

okay added it