cleanup: Improve style, formatting; use flake8, isort

This is a style cleanup, to increase code clarity and consistency,
supported by manual inspection, flake8, and isort. Most, but not
all, of these changes are to formatting, or to the way imports are
sorted and grouped (including removing unused imports).

This adds flake8, which had a configuration in setup.cfg and seems
to have previously been used, as well as isort. It configures them
for consistency with black and with the overall preexisting style.
This includes changing the line length to 88, which black uses. The
old pep8 configuration is removed, because pep8 was renamed to
pycodestyle, and the old pycodestyle is removed, because it doesn't
need to be run directly, since flake8 runs it and passes it an
appropriate configuration derived from its own. This commit sets up
flake8 and isort for local use, but it does not add them to CI.

Although flake8 detects many things black does not, they partially
overlap in their detection of overly long lines. The black rule is
more nuanced, rightly skipping over long strings in test data that
would be less clear if split across multiple lines. However, flake8
detects comments and docstrings that should be wrapped, so I didn't
turn off the flake8 rule. (Such long strings in tests are the only
cases where an inline "noqa:" suppression was needed.)

Besides tool-recommended changes, these other changes are included:

- Added the common virtualenv dirs venv and .venv to .gitignore.

- Fixed typos in comments. Made a few clarity/consistency tweaks.

- Moved a buried module docstring to top-of-file and expanded it.

- Turned a docstring that was meant as a comment into a comment.

- Made the use of initial and final newlines in and next to
  docstrings consistent within files, when doing so also made it
  more consistent with the overall style used in the project.

- Moved __all__ above other code (as PEP-8 recommends).

- Changed one __all__ from tuple to list. __all__ can be any
  sequence of strings, but the other, more prominent __all__ was a
  list (and using a list is a much more common practice).

- Replaced a few relative imports with absolute imports, since
  the overall style in this codebase is to use absolute imports.

- Changed "import X.Y as Y" to "from X import Y", which was
  sometimes also used nearby and which sometimes allowed imports to
  be consolidated.

- When "import X.Y" appeared *in X*, changed to "from X import Y".
  (Importing a submodule makes it accessible via its parent, so
  this has a similar effect. The main relevant differences, in this
  codebase, are to omit the unnecessary X.X bindings, and to make
  it easier for editors/IDEs and less experienced human programmers
  to understand what the code is doing.)

- In select cases where it seemed to noticeably improve clarity,
  changed "import X.Y" (outside X) into "from X import Y" and
  shortened the code using Y accordingly. (Mostly only when other
  imports "from X" were present and being used similarly to Y.)

- Used raw string literals for all regular expressions.

- Removed "noqa" (and applied tool-recommended fixes) in cases
  where "noqa" seems to have been added for expediency or to work
  around a combination of limitations in the old flake8
  configuration and old behavior of black (rather than to express a
  preference that the code be kept exactly as it was).

Notable omissions:

- Although flake8 and isort are configured to ignore versioneer.py,
  black must still be called with "--extend-exclude=versioneer.py".
  Customizing black's configuration would require a pyproject.toml
  file to be added, so that is deferred for now.

- To limit scope, this does not create any new CI jobs. (Such jobs
  would also not pass yet, due to the next two omissions.)

- No changes are made to conftest.py, since it may make sense to
  make deprecation-related non-style changes at the same time.

- In test_regression_143, the unused "validate" variables are kept,
  because non-stylistic considerations should determine what
  changes, if any, are to be made there.

Author: EliahKagan

.gitignore

@@ -6,9 +6,14 @@ __pycache__/
 # C extensions
 *.so
 
+# Virtual environment
+env/
+.env/
+venv/
+.venv/
+
 # Distribution / packaging
 .Python
-env/
 build/
 develop-eggs/
 dist/

development.txt

@@ -1,5 +1,7 @@
 black
 coverage
+flake8
+isort
 pandoc
 pyandoc
 pytest

docs/conf.py

@@ -12,7 +12,6 @@
 # All configuration values have a default; values that are commented out
 # serve to show the default.
 
-import sys
 import os
 
 # If extensions (or modules to document with autodoc) are in another directory,
@@ -177,8 +176,6 @@
 # If true, the index is split into individual pages for each letter.
 # html_split_index = False
 
-import os
-
 on_rtd = os.environ.get("READTHEDOCS", None) == "True"
 
 if not on_rtd:  # only import and set the theme if we're building docs locally

python_jsonschema_objects/__init__.py

@@ -1,3 +1,5 @@
+__all__ = ["ObjectBuilder", "markdown_support", "ValidationError"]
+
 import codecs
 import copy
 import json
@@ -10,16 +12,11 @@
 import six
 from jsonschema import Draft4Validator
 
-import python_jsonschema_objects.classbuilder as classbuilder
-import python_jsonschema_objects.markdown_support
-import python_jsonschema_objects.util
+from python_jsonschema_objects import classbuilder, markdown_support, util
 from python_jsonschema_objects.validators import ValidationError
 
-
 logger = logging.getLogger(__name__)
 
-__all__ = ["ObjectBuilder", "markdown_support", "ValidationError"]
-
 FILE = __file__
 
 SUPPORTED_VERSIONS = (
@@ -110,17 +107,17 @@ def build_classes(self, strict=False, named_only=False, standardize_names=True):
         Class names will be transformed using inflection by default, so names
         with spaces in the schema will be camelcased, while names without
         spaces will have internal capitalization dropped. Thus "Home Address"
-        becomes "HomeAddress", while "HomeAddress" becomes "Homeaddress" To
-        disable this behavior, pass standardize_names=False, but be aware
-        that accessing names with spaces from the namespace can be
-        problematic.
+        becomes "HomeAddress", while "HomeAddress" becomes "Homeaddress". To
+        disable this behavior, pass standardize_names=False, but be aware that
+        accessing names with spaces from the namespace can be problematic.
 
         Args:
             strict: (bool) use this to validate required fields while creating the class
-            named_only: (bool) If true, only properties with an actual title attribute will
-                be included in the resulting namespace (although all will be generated).
-            standardize_names: (bool) If true (the default), class names will be tranformed
-                by camel casing
+            named_only: (bool) If true, only properties with an actual title attribute
+                will be included in the resulting namespace (although all will be
+                generated).
+            standardize_names: (bool) If true (the default), class names will be
+                transformed by camel casing
 
         Returns:
             A namespace containing all the generated classes
@@ -129,7 +126,7 @@ def build_classes(self, strict=False, named_only=False, standardize_names=True):
         kw = {"strict": strict}
         builder = classbuilder.ClassBuilder(self.resolver)
         for nm, defn in six.iteritems(self.schema.get("definitions", {})):
-            uri = python_jsonschema_objects.util.resolve_ref_uri(
+            uri = util.resolve_ref_uri(
                 self.resolver.resolution_scope, "#/definitions/" + nm
             )
             builder.construct(uri, defn, **kw)
@@ -155,7 +152,7 @@ def build_classes(self, strict=False, named_only=False, standardize_names=True):
             elif not named_only:
                 classes[name_transform(uri.split("/")[-1])] = klass
 
-        return python_jsonschema_objects.util.Namespace.from_mapping(classes)
+        return util.Namespace.from_mapping(classes)
 
 
 if __name__ == "__main__":

python_jsonschema_objects/_version.py

@@ -11,12 +11,12 @@
 """Git implementation of _version.py."""
 
 import errno
+import functools
 import os
 import re
 import subprocess
 import sys
 from typing import Any, Callable, Dict, List, Optional, Tuple
-import functools
 
 
 def get_keywords() -> Dict[str, str]:

python_jsonschema_objects/classbuilder.py

@@ -1,23 +1,22 @@
-import python_jsonschema_objects.util as util
-import python_jsonschema_objects.validators as validators
-import python_jsonschema_objects.pattern_properties as pattern_properties
-from python_jsonschema_objects.literals import LiteralValue
-
-import copy
 import collections
+import copy
 import itertools
-import six
+import logging
 import sys
 
-import logging
+import six
 
-import python_jsonschema_objects.wrapper_types
+from python_jsonschema_objects import (
+    pattern_properties,
+    util,
+    validators,
+    wrapper_types,
+)
+from python_jsonschema_objects.literals import LiteralValue
 
 logger = logging.getLogger(__name__)
-
 logger.addHandler(logging.NullHandler())
 
-
 # Long is no longer a thing in python3.x
 if sys.version_info > (3,):
     long = int
@@ -462,7 +461,9 @@ def expand_references(self, source_uri, iterable):
         return pp
 
     def resolve_type(self, ref, source):
-        """Return a resolved type for a URI, potentially constructing one if necessary"""
+        """Return a resolved type for a URI, potentially constructing one if
+        necessary.
+        """
         uri = util.resolve_ref_uri(self.resolver.resolution_scope, ref)
         if uri in self.resolved:
             return self.resolved[uri]
@@ -556,9 +557,7 @@ def _construct(self, uri, clsdata, parent=(ProtocolBase,), **kw):
         elif clsdata.get("type") == "array" and "items" in clsdata:
             clsdata_copy = {}
             clsdata_copy.update(clsdata)
-            self.resolved[
-                uri
-            ] = python_jsonschema_objects.wrapper_types.ArrayWrapper.create(
+            self.resolved[uri] = wrapper_types.ArrayWrapper.create(
                 uri,
                 item_constraint=clsdata_copy.pop("items"),
                 classbuilder=self,
@@ -684,7 +683,7 @@ def _build_object(self, nm, clsdata, parents, **kw):
                         constraints["strict"] = kw.get("strict")
                         propdata = {
                             "type": "array",
-                            "validator": python_jsonschema_objects.wrapper_types.ArrayWrapper.create(
+                            "validator": wrapper_types.ArrayWrapper.create(
                                 nm, item_constraint=typ, **constraints
                             ),
                         }
@@ -705,7 +704,7 @@ def _build_object(self, nm, clsdata, parents, **kw):
                             constraints["strict"] = kw.get("strict")
                             propdata = {
                                 "type": "array",
-                                "validator": python_jsonschema_objects.wrapper_types.ArrayWrapper.create(
+                                "validator": wrapper_types.ArrayWrapper.create(
                                     uri, item_constraint=typ, **constraints
                                 ),
                             }
@@ -716,7 +715,7 @@ def _build_object(self, nm, clsdata, parents, **kw):
                             constraints["strict"] = kw.get("strict")
                             propdata = {
                                 "type": "array",
-                                "validator": python_jsonschema_objects.wrapper_types.ArrayWrapper.create(
+                                "validator": wrapper_types.ArrayWrapper.create(
                                     uri, item_constraint=typ, **constraints
                                 ),
                             }

python_jsonschema_objects/descriptors.py

@@ -1,5 +1,5 @@
-from . import validators, util, wrapper_types
-from .classbuilder import ProtocolBase, TypeProxy, TypeRef
+from python_jsonschema_objects import util, validators, wrapper_types
+from python_jsonschema_objects.classbuilder import ProtocolBase, TypeProxy, TypeRef
 
 
 class AttributeDescriptor(object):
@@ -58,7 +58,7 @@ def __set__(self, obj, val):
                         ok = True
                         break
                 elif util.safe_issubclass(typ, ProtocolBase):
-                    # force conversion- thus the val rather than validator assignment
+                    # Force conversion- thus the val rather than validator assignment.
                     try:
                         val = typ(**util.coerce_for_expansion(val))
                         val.validate()
@@ -80,8 +80,8 @@ def __set__(self, obj, val):
                         break
                 elif isinstance(typ, TypeProxy):
                     try:
-                        # handle keyword expansion according to expected types
-                        # using keywords like oneOf, value can be an object, array or literal
+                        # Handle keyword expansion according to expected types. Using
+                        # keywords like oneOf, value can be an object, array or literal.
                         val = util.coerce_for_expansion(val)
                         if isinstance(val, dict):
                             val = typ(**val)
@@ -106,7 +106,7 @@ def __set__(self, obj, val):
             val.validate()
 
         elif util.safe_issubclass(info["type"], wrapper_types.ArrayWrapper):
-            # An array type may have already been converted into an ArrayValidator
+            # An array type may have already been converted into an ArrayValidator.
             val = info["type"](val)
             val.validate()
 
@@ -115,7 +115,7 @@ def __set__(self, obj, val):
                 validator = info["type"](val)
                 validator.validate()
                 if validator._value is not None:
-                    # This allows setting of default Literal values
+                    # This allows setting of default Literal values.
                     val = validator
 
         elif util.safe_issubclass(info["type"], ProtocolBase):

python_jsonschema_objects/literals.py

@@ -1,10 +1,10 @@
-from python_jsonschema_objects import util
-from python_jsonschema_objects import validators
 import functools
-import logging
-import six
 import operator
 
+import six
+
+from python_jsonschema_objects import util, validators
+
 
 def MakeLiteral(name, typ, value, **properties):
     properties.update({"type": typ})
@@ -76,7 +76,8 @@ def __str__(self):
     def validate(self):
         info = self.propinfo("__literal__")
 
-        # TODO: this duplicates logic in validators.ArrayValidator.check_items; unify it.
+        # TODO: this duplicates logic in validators.ArrayValidator.check_items;
+        # unify it.
         for param, paramval in sorted(
             six.iteritems(info), key=lambda x: x[0].lower() != "type"
         ):

python_jsonschema_objects/markdown_support.py

@@ -1,8 +1,8 @@
+import re
+
 import markdown
 from markdown.extensions import Extension
 from markdown.preprocessors import Preprocessor
-import re
-import json
 
 try:
     from markdown import __version_info__ as markdown_version_info

python_jsonschema_objects/pattern_properties.py

@@ -1,14 +1,11 @@
-import six
-import re
-import python_jsonschema_objects.validators as validators
-import python_jsonschema_objects.util as util
-from python_jsonschema_objects.literals import MakeLiteral
-
 import collections
-
 import logging
+import re
+
+import six
 
-import python_jsonschema_objects.wrapper_types
+from python_jsonschema_objects import util, validators, wrapper_types
+from python_jsonschema_objects.literals import MakeLiteral
 
 logger = logging.getLogger(__name__)
 
@@ -66,9 +63,7 @@ def _make_type(self, typ, val):
         if util.safe_issubclass(typ, cb.ProtocolBase):
             return typ(**util.coerce_for_expansion(val))
 
-        if util.safe_issubclass(
-            typ, python_jsonschema_objects.wrapper_types.ArrayWrapper
-        ):
+        if util.safe_issubclass(typ, wrapper_types.ArrayWrapper):
             return typ(val)
 
         if isinstance(typ, cb.TypeProxy):

python_jsonschema_objects/util.py

@@ -1,6 +1,12 @@
-import six
+"""Utility and namespace module."""
+
+__all__ = ["Namespace", "as_namespace"]
+
 import copy
 import json
+from collections.abc import Mapping, Sequence
+
+import six
 
 
 class lazy_format(object):
@@ -22,7 +28,8 @@ def safe_issubclass(x, y):
     in the underlying implementation throwing TypeError's from trying to
     memoize the result- 'object' isn't a usable weakref target at that level.
     Unfortunately this gets exposed all the way up to our code; thus a
-    'safe' version of the function."""
+    'safe' version of the function.
+    """
     try:
         return issubclass(x, y)
     except TypeError:
@@ -34,16 +41,16 @@ def coerce_for_expansion(mapping):
 
     In py2.7, the value must be a dictionary- thus a as_dict() method
     will be invoked if available.  In py3k, the raw mapping is returned
-    unmodified."""
+    unmodified.
+    """
     if six.PY2 and hasattr(mapping, "as_dict"):
         return mapping.as_dict()
     return mapping
 
 
 class ProtocolJSONEncoder(json.JSONEncoder):
     def default(self, obj):
-        from python_jsonschema_objects import classbuilder
-        from python_jsonschema_objects import wrapper_types
+        from python_jsonschema_objects import classbuilder, wrapper_types
 
         if isinstance(obj, classbuilder.LiteralValue):
             return obj._value
@@ -112,13 +119,6 @@ def resolve_ref_uri(base, ref):
     return uri
 
 
-"""namespace module"""
-
-__all__ = ("Namespace", "as_namespace")
-
-from collections.abc import Mapping, Sequence
-
-
 class _Dummy:
     pass
 
@@ -129,12 +129,10 @@ class _Dummy:
 
 
 class Namespace(dict):
-
     """A dict subclass that exposes its items as attributes.
 
     Warning: Namespace instances do not have direct access to the
     dict methods.
-
     """
 
     def __init__(self, obj={}):