Migrated to Python 3 (only)

commit f69c25a
Author: sourcesimian <[email protected]>
Date:   Mon Dec 20 20:21:10 2021 +0200

    Migrate to Python 3

commit 9d5040b
Author: sourcesimian <[email protected]>
Date:   Sat Jun 9 14:13:25 2018 +0200

    : Added unittests for bake and dictfilesystem

commit 20caca1
Author: sourcesimian <[email protected]>
Date:   Fri Jun 8 23:58:59 2018 +0200

    : Python 3 compatibility

Author: sourcesimian

.gitignore

@@ -1,7 +1,9 @@
 *.pyc
 .idea/
-.cache/
+.cache*/
 reports/
 dist/
 pybake.egg-info/
 virtualenv/
+venv/
+.pytest_cache/

Makefile

@@ -1,15 +1,11 @@
 
+
 develop:
-	virtualenv --no-site-packages --python /usr/bin/python2.7 virtualenv
-	{ \
-	    . ./virtualenv/bin/activate; \
-	    curl https://bootstrap.pypa.io/get-pip.py | python; \
-	    pip install pytest flake8; \
-	}
+	python3 -m venv venv
 
 
 clean:
-	git clean -dfx
+	git clean -dfxn | grep 'Would remove' | awk '{print $3}' | grep -v -e '^.idea' -e '^.cache' | xargs rm -rf
 
 
 check:
@@ -20,11 +16,11 @@ test:
 	pytest ./tests/ -vvv --junitxml=./reports/unittest-results.xml
 
 
-to_pypi_test:
-	python setup.py register -r pypitest
-	python setup.py sdist upload -r pypitest
+to_pypi_test: test
+	python -m build
+	twine upload -r testpypi dist/*
 
 
-to_pypi_live:
-	python setup.py register -r pypi
-	python setup.py sdist upload -r pypi
+to_pypi_live: test
+	python -m build
+	twine upload dist/*

README.md

@@ -1,3 +1,116 @@
-# PyBake
+PyBake <!-- omit in toc -->
+===
 
-Build single file Python scripts with builtin frozen file system
+***Create single file standalone Python scripts with builtin frozen file system***
+
+- [Purpose](#purpose)
+- [Usage](#usage)
+  - [Bake Script](#bake-script)
+- [Advanced](#advanced)
+  - [Tracebacks](#tracebacks)
+  - [Inspection Server](#inspection-server)
+  - [Content](#content)
+  - [Frozen File Systems](#frozen-file-systems)
+- [License](#license)
+
+# Purpose
+PyBake can bundle an entire Python project including modules and data files into a single Python file, to provide a "standalone executable" style utility. PyBake supports pure Python modules only, thus PyBakes are multi-platform.
+
+PyBakes just run and don't need an installation so are a great way of rapidly distributing tooling or utility scripts, yet the development can still be done using a formal module hierarchy.
+
+The intention of PyBake is to be lightweight and to remain pure Python. There are several other "standalone executable" Python projects, with differing options and capabilities:
+* [PyInstaller](https://www.pyinstaller.org/)
+* [py2exe](https://www.py2exe.org/)
+* [cx_Freeze](https://marcelotduarte.github.io/cx_Freeze)
+* [py2app](https://github.com/ronaldoussoren/py2app/blob/master/README.rst)
+
+
+
+
+# Usage
+The current usage idiom is to create a bake script that is run when you wish to produce a release of your utility.
+## Bake Script
+This bake script includes an optional header to provide version information and setup instructions. The footer is used to invoke the desired entry point. The footer can be
+omitted should you wish your PyBake to simply act as a module that can be imported into other scripts, e.g.:
+```
+#!/usr/bin/env python3
+from pybake import PyBake
+
+HEADER = '''\
+################################################################################
+##  my-util - Helper for ...                                                  ##
+##  Setup: Save this entire script to a file called "my-util.py".             ##
+##         Make it executable, e.g.: `chmod +x ./my-util.py`.                 ##
+##         Run `./my-util.py --help`                                          ##
+##  Version:   0.1                                                            ##
+################################################################################
+'''
+
+FOOTER = '''\
+if __name__ == '__main__':                                                    ##
+    from my_util.main import main                                             ##
+    exit(main())                                                              ##
+'''
+
+pb = PyBake(HEADER, FOOTER)
+import my_util
+pb.add_module(my_util)
+
+with open('./data.json', 'rt') as fh:
+    pb.add_file(('my_util', 'data.json'), fh)
+
+pb.write_bake('my-util.py')
+```
+
+Then just run `my-util.py`.
+
+# Advanced
+## Tracebacks
+A PyBake maintains a full representation of source paths and line numbers in Tracebacks. So it is easy to track down the source of any unhandled exceptions. For example if there was a `KeyError` in `main.py` of your utility, the Traceback would appear as follows. The path points to the PyBake path, and then the path from the frozen filesystem is suffixed.
+```
+Traceback (most recent call last):
+  File "/home/user/./my-util.py", line 132, in <module>
+    exit(main())                                                              ##
+  File "/home/user/./my-util.py/my_util/main.py", line 3, in main
+KeyError: 'Oops!'
+```
+
+## Inspection Server
+Sometimes you may want to inspect the contents of your PyBake. This can be done by running your PyBake with the `--pybake-server` argument. It will run a small HTTP server that serves on `localhost:8080`. Open your web browser to http://localhost:8080 from where you can browse the contents. An additional argument will be interpreted as a port.
+
+This feature is also a good way to calm the minds of your more suspicious colleagues, who think you're up to something nefarious because you've "obfuscated" your code. And then after lunch they happily `apt-get` install some binary on their machine without first reading the machine code ¯\_(ツ)_/¯
+
+## Content
+The content of a PyBake is pure Python. However, if you glance at the source you might think that it is some form of executable. However, taking a closer look at the head will show, e.g.:
+```
+#!/usr/bin/env python3
+################################################################################
+##  my-util - Helper for ...                                                  ##
+##  ...
+################################################################################
+_='''eJzVPGuT2kiSf6XDGxv2xPV6QTQ+MxHzAWSkRtB4gEYIeRwdSAIkkISmxUtszH+/zCqp9CrRjHf
+```
+Then many of lines of "garbage", e.g.:
+```
+MzsXtdQdhkOqRle/Myqqv77wg2r8cbg5JtI5/DdmvOIHv+B++fAz2ztFfxx/j9cFZb1ZH//DhfZRYq93
+```
+And the tail will show. e.g.:
+```
+...
+WgtFLYPu09hG9EDwjGF8MG9ei/fHH9/8BdRTTUQ=='''
+import binascii, json, zlib                                                   ##
+_ = json.loads(zlib.decompress(binascii.a2b_base64(_))); exec(_[0], globals())##
+if __name__ == '__main__':                                                    ##
+    from my_util.main import main                                             ##
+    exit(main())                                                              ##
+###########################################################################END##
+```
+As you can see a PyBake is simply a very short Python script with a giant zlib compressed Base 64 encoded data structure assigned to a string called `-`.
+
+## Frozen File Systems
+As mentioned PyBake is pure Python and serves as a demonstration of the awesome builtin capabilities of the Python standard libraries. If you are interested to understand more, start reading the source in [launcher.py](./pybake/launcher.py) and follow the `DictFileSystem`.
+
+
+# License
+
+In the spirit of the Hackers of the [Tech Model Railroad Club](https://en.wikipedia.org/wiki/Tech_Model_Railroad_Club) from the [Massachusetts Institute of Technology](https://en.wikipedia.org/wiki/Massachusetts_Institute_of_Technology), who gave us all so very much to play with. The license is [MIT](LICENSE).

pybake/__init__.py

@@ -0,0 +1,5 @@
+
+
+__all__ = ['PyBake']
+
+from pybake.pybake import PyBake

pybake/abstractimporter.py

@@ -1,4 +1,6 @@
-import imp
+
+
+import types
 import os
 import sys
 
@@ -21,7 +23,7 @@ def __exit__(self, exc_type, exc_val, exc_tb):
         self.uninstall()
 
     def install(self):
-        sys.meta_path.append(self)
+        sys.meta_path.insert(0, self)
 
     def uninstall(self):
         try:
@@ -64,7 +66,7 @@ def find_module(self, fullname, path=None):
     def load_module(self, fullname):
         full_path = self._full_path(fullname)
 
-        mod = sys.modules.setdefault(fullname, imp.new_module(fullname))
+        mod = sys.modules.setdefault(fullname, types.ModuleType(fullname))
         mod.__file__ = full_path
         mod.__loader__ = self
 
@@ -76,18 +78,16 @@ def load_module(self, fullname):
             mod.__package__ = '.'.join(fullname.split('.')[:-1])
         source = self._read_file(full_path)
         try:
-            exec compile(source, full_path, 'exec') in mod.__dict__
+            exec(compile(source, full_path, 'exec'), mod.__dict__)
         except ImportError:
             exc_info = sys.exc_info()
-            raise exc_info[0], "%s, while importing '%s'" % (exc_info[1], fullname), exc_info[2]
-            # raise exc_info[0]  from ex
+            exc_info1 = ImportError("%s, while importing '%s'" % (exc_info[1], fullname))
+            reraise(exc_info[0], exc_info1, exc_info[2])
         except Exception:
-            orig_exc_type, exc, tb = sys.exc_info()
-            exc_info = (ImportError, exc, tb)
-            raise exc_info[0], "%s: %s, while importing '%s'" % (orig_exc_type.__name__,
-                                                                 exc_info[1],
-                                                                 fullname), exc_info[2]
-            # raise exc_info[0] from ex
+            exc_info = sys.exc_info()
+            exc_info1 = ImportError("%s: %s, while importing '%s'" % (exc_info[0].__name__,
+                                                                      exc_info[1], fullname))
+            reraise(ImportError, exc_info1, exc_info[2])
         self._add_module(fullname)
         return mod
 
@@ -96,13 +96,28 @@ def get_source(self, fullname):
         return self._read_file(full_path)
 
     def is_package(self, fullname):
-        print '!!!! is_package',
+        path = self._full_path(fullname)
+        if path:
+            return path.endswith('__init__.py')
+        return False
 
     def get_code(self, fullname):
-        print '!!!! get_code',
+        print('!!!! get_code')
 
     def get_data(self, path):
-        print '!!!! get_data',
+        print('!!!! get_data')
 
     def get_filename(self, fullname):
-        print '!!!! get_filename',
+        return self._full_path(fullname)
+
+
+def reraise(tp, value, tb=None):
+    try:
+        if value is None:
+            value = tp()
+        if value.__traceback__ is not tb:
+            raise value.with_traceback(tb)
+        raise value
+    finally:
+        value = None
+        tb = None

pybake/blobserver.py

@@ -19,8 +19,7 @@ def run(self, address='8080'):
             port = address
         port = int(port)
 
-        from SimpleHTTPServer import SimpleHTTPRequestHandler
-        from BaseHTTPServer import HTTPServer
+        from http.server import SimpleHTTPRequestHandler, HTTPServer
 
         class Server(HTTPServer):
             def __init__(self, *args, **kwargs):

pybake/dictfilesystem.py

@@ -8,7 +8,7 @@ class DictFileSystem(object):
 
     def __init__(self, base_dir=None, dict_tree=None):
         self._base_dir = self._normalise_path(base_dir or '/pybake/root')
-        self._dict_tree = dict_tree or {}
+        self._dict_tree = {} if dict_tree is None else dict_tree
         try:
             self._base_stat = os.stat(self._base_dir)
         except OSError:
@@ -17,7 +17,7 @@ def __init__(self, base_dir=None, dict_tree=None):
     def tpath(self, path):
         path = self._normalise_path(path)
         try:
-            if isinstance(path, basestring) and path.startswith(self._base_dir):
+            if isinstance(path, str) and path.startswith(self._base_dir):
                 subpath = path[len(self._base_dir):]
                 # if not subpath:
                 #     return None
@@ -52,7 +52,7 @@ def _get_node(self, tpath):
         raise IOError(errno.ENOENT, "No such file or directory", '%s/%s' % (self._base_dir, os.sep.join(tpath)))
 
     def listdir(self, tpath):
-        return self._get_node(tpath).keys()
+        return sorted(self._get_node(tpath).keys())
 
     def read(self, tpath):
         type, content = self._get_node(tpath)
@@ -105,7 +105,9 @@ def stat(self, tpath):
 
     def get_dict_tree(self):
         def encode(type, content):
-            return ('base64', binascii.b2a_base64(content))
+            if isinstance(content, str):
+                content = content.encode('utf-8')
+            return ('base64', binascii.b2a_base64(content).decode('utf-8'))
             # try:
             #     content.decode('ascii')
             #     return ('raw', content)
@@ -121,7 +123,8 @@ def walk(src):
                     dst[key] = encode(*src[key])
             return dst
 
-        return walk(self._dict_tree)
+        tree = walk(self._dict_tree)
+        return tree
 
     @staticmethod
     def _normalise_path(filename):

pybake/dictfilesystembuilder.py

@@ -16,6 +16,8 @@ def add_module(self, module, base=()):
 
             if module.__package__ is not None:
                 package = tuple(module.__package__.split('.'))
+            else:
+                package = tuple(module.__name__.split('.'))
         elif os.path.exists(module):
             if os.path.isfile(module):
                 basedir = os.path.dirname(module)

pybake/dictfilesysteminterceptor.py

@@ -1,14 +1,16 @@
+
+
 import errno
 import os
-from StringIO import StringIO
+from io import BytesIO
 
-from pybake.filesysteminterceptor import FileSystemInterceptor
+from pybake.filesysteminterceptor import FileSystemInterceptor, BUILTINS_OPEN
 
 
 class DictFileSystemInterceptor(FileSystemInterceptor):
     def __init__(self, reader):
         intercept_list = [
-            '__builtin__.open',
+            BUILTINS_OPEN,
             'os.path.isfile',
             'os.path.isdir',
             'os.path.exists',
@@ -21,10 +23,13 @@ def __init__(self, reader):
         self._reader = reader
         self._fileno = FileNo()
 
-    def _builtin_open(self, path, mode='r'):
+    def _builtins_open(self, *args, **kwargs):
+        return self._builtin_open(*args, **kwargs)
+
+    def _builtin_open(self, path, mode='r', *args, **kwargs):
         tpath = self._reader.tpath(path)
         if tpath is None:
-            return self._oldhooks['__builtin__.open'](path, mode)
+            return self._oldhooks[BUILTINS_OPEN](path, mode, *args, **kwargs)
         if 'w' in mode:
             raise IOError(errno.EROFS, 'Read-only file system', path)
         content = self._reader.read(tpath)
@@ -119,11 +124,11 @@ def close_path(self, path):
             pass
 
 
-class FrozenFile(StringIO):
+class FrozenFile(BytesIO):
     def __init__(self, fileno, path, content):
         self._fileno = fileno
         self._path = path
-        StringIO.__init__(self, content)
+        BytesIO.__init__(self, content)
 
     def __enter__(self):
         return self