Merge pull request #511 from splunk/release/1.7.3

Release/1.7.3

Author: ashah-splunk

.env

@@ -11,6 +11,6 @@ scheme=https
 # Your version of Splunk (default: 6.2)
 version=9.0
 # Bearer token for authentication
-#bearerToken="<Bearer-token>"
+#splunkToken="<Bearer-token>"
 # Session key for authentication
-#sessionKey="<Session-Key>"
+#token="<Session-Key>"

.github/dependabot.yaml

@@ -0,0 +1,7 @@
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    target-branch: "master"
+    schedule:
+      interval: "weekly"

.github/workflows/release.yml

@@ -29,12 +29,11 @@ jobs:
         run: |
           rm -rf ./docs/_build
           tox -e docs
-          cd ./docs/_build/html && zip -r ../docs_html.zip . -x ".*" -x "__MACOSX"
       - name : Docs Upload
         uses: actions/upload-artifact@v3
         with:
-          name: apidocs
-          path: docs/_build/docs_html.zip
+          name: python_sdk_docs
+          path: docs/_build/html
       # Test upload
       # - name: Publish package to TestPyPI
       #   uses: pypa/gh-action-pypi-publish@master

.github/workflows/test.yml

@@ -13,6 +13,7 @@ jobs:
           - ubuntu-latest
         python: [ 2.7, 3.7 ]
         splunk-version:
+          - "8.1"
           - "8.2"
           - "latest"
       fail-fast: false

CHANGELOG.md

@@ -1,5 +1,20 @@
 # Splunk Enterprise SDK for Python Changelog
 
+## Version 1.7.3
+
+### Bug fixes
+* [#493](https://github.com/splunk/splunk-sdk-python/pull/493) Fixed file permission for event_writer.py file [[issue#487](https://github.com/splunk/splunk-sdk-python/issues/487)]
+* [#500](https://github.com/splunk/splunk-sdk-python/pull/500) Replaced index_field with accelerated_field for kvstore [[issue#497](https://github.com/splunk/splunk-sdk-python/issues/497)]
+* [#502](https://github.com/splunk/splunk-sdk-python/pull/502) Updated check for IPv6 addresses
+
+### Minor changes
+* [#490](https://github.com/splunk/splunk-sdk-python/pull/490) Added ACL properties update feature
+* [#495](https://github.com/splunk/splunk-sdk-python/pull/495) Added Splunk 8.1 in GitHub Actions Matrix
+* [#485](https://github.com/splunk/splunk-sdk-python/pull/485) Added test case for cookie persistence
+* [#503](https://github.com/splunk/splunk-sdk-python/pull/503) README updates on accessing "service" instance in CSC and ModularInput apps
+* [#504](https://github.com/splunk/splunk-sdk-python/pull/504) Updated authentication token names in docs to reduce confusion
+* [#494](https://github.com/splunk/splunk-sdk-python/pull/494) Reuse splunklib.__version__ in handler.request 
+
 ## Version 1.7.2
 
 ### Minor changes

README.md

@@ -3,7 +3,7 @@
 
 # The Splunk Enterprise Software Development Kit for Python
 
-#### Version 1.7.2
+#### Version 1.7.3
 
 The Splunk Enterprise Software Development Kit (SDK) for Python contains library code designed to enable developers to build applications using the Splunk platform.
 
@@ -30,7 +30,7 @@ Here's what you need to get going with the Splunk Enterprise SDK for Python.
 
 * Splunk Enterprise 9.0 or 8.2
 
-    The Splunk Enterprise SDK for Python has been tested with Splunk Enterprise 9.0 and 8.2
+    The Splunk Enterprise SDK for Python has been tested with Splunk Enterprise 9.0, 8.2 and 8.1
 
   If you haven't already installed Splunk Enterprise, download it [here](http://www.splunk.com/download). 
   For more information, see the Splunk Enterprise [_Installation Manual_](https://docs.splunk.com/Documentation/Splunk/latest/Installation).
@@ -111,9 +111,9 @@ here is an example of .env file:
     # Your version of Splunk Enterprise
     version=9.0
     # Bearer token for authentication
-    #bearerToken=<Bearer-token>
+    #splunkToken=<Bearer-token>
     # Session key for authentication
-    #sessionKey=<Session-Key>
+    #token=<Session-Key>
 
 #### SDK examples
 
@@ -209,7 +209,39 @@ class GeneratorTest(GeneratingCommand):
         checkpoint_dir = inputs.metadata["checkpoint_dir"]
 ```
 
-#### Optional:Set up logging for splunklib
+### Access service object in Custom Search Command & Modular Input apps
+
+#### Custom Search Commands
+* The service object is created from the Splunkd URI and session key passed to the command invocation the search results info file.
+* Service object can be accessed using `self.service` in `generate`/`transform`/`stream`/`reduce` methods depending on the Custom Search Command.
+* For Generating Custom Search Command
+  ```python
+    def generate(self):
+        # other code
+        
+        # access service object that can be used to connect Splunk Service
+        service = self.service
+        # to get Splunk Service Info
+        info = service.info
+  ```
+
+ 
+
+#### Modular Inputs app:
+* The service object is created from the Splunkd URI and session key passed to the command invocation on the modular input stream respectively.
+* It is available as soon as the `Script.stream_events` method is called.
+```python
+    def stream_events(self, inputs, ew):
+        # other code
+        
+        # access service object that can be used to connect Splunk Service
+        service = self.service
+        # to get Splunk Service Info
+        info = service.info
+```
+
+
+### Optional:Set up logging for splunklib
 + The default level is WARNING, which means that only events of this level and above will be visible
 + To change a logging level we can call setup_logging() method and pass the logging level as an argument.
 + Optional: we can also pass log format and date format string as a method argument to modify default format

scripts/templates/env.template

@@ -11,6 +11,6 @@ scheme=$scheme
 # Your version of Splunk (default: 6.2)
 version=$version
 # Bearer token for authentication
-#bearerToken=<Bearer-token>
+#splunkToken=<Bearer-token>
 # Session key for authentication
-#sessionKey=<Session-Key>
+#token=<Session-Key>

splunklib/__init__.py

@@ -31,5 +31,5 @@ def setup_logging(level, log_format=DEFAULT_LOG_FORMAT, date_format=DEFAULT_DATE
                         format=log_format,
                         datefmt=date_format)
 
-__version_info__ = (1, 7, 2)
+__version_info__ = (1, 7, 3)
 __version__ = ".".join(map(str, __version_info__))

splunklib/binding.py

@@ -39,6 +39,7 @@
 from io import BytesIO
 from xml.etree.ElementTree import XML
 
+from splunklib import __version__
 from splunklib import six
 from splunklib.six.moves import urllib
 
@@ -346,7 +347,8 @@ def _authority(scheme=DEFAULT_SCHEME, host=DEFAULT_HOST, port=DEFAULT_PORT):
             "http://splunk.utopia.net:471"
 
     """
-    if ':' in host:
+    # check if host is an IPv6 address and not enclosed in [ ]
+    if ':' in host and not (host.startswith('[') and host.endswith(']')):
         # IPv6 addresses must be enclosed in [ ] in order to be well
         # formed.
         host = '[' + host + ']'
@@ -1434,7 +1436,7 @@ def request(url, message, **kwargs):
         head = {
             "Content-Length": str(len(body)),
             "Host": host,
-            "User-Agent": "splunk-sdk-python/1.7.2",
+            "User-Agent": "splunk-sdk-python/%s" % __version__,
             "Accept": "*/*",
             "Connection": "Close",
         } # defaults

splunklib/client.py

@@ -1216,6 +1216,36 @@ def reload(self):
         self.post("_reload")
         return self
 
+    def acl_update(self, **kwargs):
+        """To update Access Control List (ACL) properties for an endpoint.
+
+        :param kwargs: Additional entity-specific arguments (required).
+
+            - "owner" (``string``): The Splunk username, such as "admin". A value of "nobody" means no specific user (required).
+
+            - "sharing" (``string``): A mode that indicates how the resource is shared. The sharing mode can be "user", "app", "global", or "system" (required).
+
+        :type kwargs: ``dict``
+
+        **Example**::
+
+            import splunklib.client as client
+            service = client.connect(...)
+            saved_search = service.saved_searches["name"]
+            saved_search.acl_update(sharing="app", owner="nobody", app="search", **{"perms.read": "admin, nobody"})
+        """
+        if "body" not in kwargs:
+            kwargs = {"body": kwargs}
+
+        if "sharing" not in kwargs["body"]:
+            raise ValueError("Required argument 'sharing' is missing.")
+        if "owner" not in kwargs["body"]:
+            raise ValueError("Required argument 'owner' is missing.")
+
+        self.post("acl", **kwargs)
+        self.refresh()
+        return self
+
     @property
     def state(self):
         """Returns the entity's state record.
@@ -3679,24 +3709,31 @@ class KVStoreCollections(Collection):
     def __init__(self, service):
         Collection.__init__(self, service, 'storage/collections/config', item=KVStoreCollection)
 
-    def create(self, name, indexes = {}, fields = {}, **kwargs):
+    def __getitem__(self, item):
+        res = Collection.__getitem__(self, item)
+        for k, v in res.content.items():
+            if "accelerated_fields" in k:
+                res.content[k] = json.loads(v)
+        return res
+
+    def create(self, name, accelerated_fields={}, fields={}, **kwargs):
         """Creates a KV Store Collection.
 
         :param name: name of collection to create
         :type name: ``string``
-        :param indexes: dictionary of index definitions
-        :type indexes: ``dict``
+        :param accelerated_fields: dictionary of accelerated_fields definitions
+        :type accelerated_fields: ``dict``
         :param fields: dictionary of field definitions
         :type fields: ``dict``
         :param kwargs: a dictionary of additional parameters specifying indexes and field definitions
         :type kwargs: ``dict``
 
         :return: Result of POST request
         """
-        for k, v in six.iteritems(indexes):
+        for k, v in six.iteritems(accelerated_fields):
             if isinstance(v, dict):
                 v = json.dumps(v)
-            kwargs['index.' + k] = v
+            kwargs['accelerated_fields.' + k] = v
         for k, v in six.iteritems(fields):
             kwargs['field.' + k] = v
         return self.post(name=name, **kwargs)
@@ -3710,18 +3747,20 @@ def data(self):
         """
         return KVStoreCollectionData(self)
 
-    def update_index(self, name, value):
-        """Changes the definition of a KV Store index.
+    def update_accelerated_field(self, name, value):
+        """Changes the definition of a KV Store accelerated_field.
 
-        :param name: name of index to change
+        :param name: name of accelerated_fields to change
         :type name: ``string``
-        :param value: new index definition
-        :type value: ``dict`` or ``string``
+        :param value: new accelerated_fields definition
+        :type value: ``dict``
 
         :return: Result of POST request
         """
         kwargs = {}
-        kwargs['index.' + name] = value if isinstance(value, six.string_types) else json.dumps(value)
+        if isinstance(value, dict):
+            value = json.dumps(value)
+        kwargs['accelerated_fields.' + name] = value
         return self.post(**kwargs)
 
     def update_field(self, name, value):

splunklib/modularinput/event_writer.py

@@ -190,6 +190,12 @@ def test_ipv6_host(self):
                 host="2001:0db8:85a3:0000:0000:8a2e:0370:7334"),
             "https://[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:8089")
 
+    def test_ipv6_host_enclosed(self):
+        self.assertEqual(
+            binding._authority(
+                host="[2001:0db8:85a3:0000:0000:8a2e:0370:7334]"),
+            "https://[2001:0db8:85a3:0000:0000:8a2e:0370:7334]:8089")
+
     def test_all_fields(self):
         self.assertEqual(
             binding._authority(
@@ -491,6 +497,56 @@ def test_handlers(self):
                 body = context.get(path).body.read()
                 self.assertTrue(isatom(body))
 
+def urllib2_insert_cookie_handler(url, message, **kwargs):
+    method = message['method'].lower()
+    data = message.get('body', b"") if method == 'post' else None
+    headers = dict(message.get('headers', []))
+    req = Request(url, data, headers)
+    try:
+        # If running Python 2.7.9+, disable SSL certificate validation
+        if sys.version_info >= (2, 7, 9):
+            response = urlopen(req, context=ssl._create_unverified_context())
+        else:
+            response = urlopen(req)
+    except HTTPError as response:
+        pass # Propagate HTTP errors via the returned response message
+
+    # Mimic the insertion of 3rd party cookies into the response.
+    # An example is "sticky session"/"insert cookie" persistence
+    # of a load balancer for a SHC.
+    header_list = [(k, v) for k, v in response.info().items()] 
+    header_list.append(("Set-Cookie", "BIGipServer_splunk-shc-8089=1234567890.12345.0000; path=/; Httponly; Secure"))
+    header_list.append(("Set-Cookie", "home_made=yummy"))
+    
+    return {
+        'status': response.code,
+        'reason': response.msg,
+        'headers': header_list,
+        'body': BytesIO(response.read())
+    }
+
+class TestCookiePersistence(testlib.SDKTestCase):
+    # Verify persistence of 3rd party inserted cookies.
+    def test_3rdPartyInsertedCookiePersistence(self):
+        paths = ["/services", "authentication/users",
+                 "search/jobs"]
+        logging.debug("Connecting with urllib2_insert_cookie_handler %s", urllib2_insert_cookie_handler)
+        context = binding.connect(
+            handler=urllib2_insert_cookie_handler,
+            **self.opts.kwargs)
+
+        persisted_cookies = context.get_cookies()
+
+        splunk_token_found = False
+        for k, v in persisted_cookies.items():
+            if k[:8] == "splunkd_":
+                splunk_token_found = True
+                break
+
+        self.assertEqual(splunk_token_found, True)
+        self.assertEqual(persisted_cookies['BIGipServer_splunk-shc-8089'], "1234567890.12345.0000")
+        self.assertEqual(persisted_cookies['home_made'], "yummy")
+
 @pytest.mark.smoke
 class TestLogout(BindingTestCase):
     def test_logout(self):