Advanced Python Functionality

API Settings

The Python SDK provides a few methods that affect interactions with the ThreatConnect API. The default settings should work in most cases.

Logging

Example of Python SDK calling log-file and debug level:

# set a destination log path and logging level
tc.set_tcl_file('log/tc.log', 'debug')
# set the console logging level
tc.set_tcl_console_level('critical')

The Python SDK allows for the setting of the log-file location and debug level. The level on the console logging can be set as well. The default logging level for each is critical.

Activity Log

Enabling the Activity Log:

tc.set_activity_log(True)

Disabling the Activity Log:

tc.set_activity_log(False)

The ThreatConnect platform tracks all activity by users, including API accounts. When using the API to create thousands of Indicators, the Activity Log could generate excessive data. Activity tracing is disabled by default in the Python SDK. This feature can be turned on by calling the tc.set_activity_log() method, passing True as the argument. To disable tracking again during the same execution period, call the tc.set_activity_log() method once more, passing False.

API Request Timeout

This timeout value can be changed by passing the new timeout value, in seconds, to the tc.set_api_request_timeout() method.

tc.set_api_request_timeout(15)

The Python SDK uses the Request module for communicating to the API. To prevent script from hanging on a bad socket, there is a timeout value set to 30 seconds by default.

API Retries/Sleep Period

To change the default sleep period, call the set_api_sleep() method, passing an Integer for the number of seconds to sleep.

tc.set_api_retries(3)
tc.set_api_sleep(30)

If the Python SDK loses network connectivity to the API server, it will automatically retry the connection.

The Python SDK has a default of 5 retries, with a default 59-second sleep between retries before a RuntimeError is raised. To change the default retry value, call the set_api_retries() method, passing an Integer for the number of retries.

Note

There is a maximum window of 5 minutes before the API will reject the HMAC (hash message authentication code) header due to a time mismatch.

API Result Limit

The ThreatConnect API supports a maximum of 10,000 results to be returned per API call during pagination. The Python SDK is configured for a default of 200 results per API request. To change the default value, call the set_api_result_limit() method, passing an Integer between 1 and 10,000. The higher the number, the less API calls will be made, but in some cases, a lower number is required due to network limitations.

tc.set_api_result_limit(10,000)

Proxies

Proxy Setting (No Authentication)

tc.set_proxies('10.10.10.10', 8443)

Proxy Setting (Authentication Provided)

tc.set_proxies('10.10.10.10', 8443, 'proxy_user', 'password123')

In some environments, the server running the Python SDK does not have the required Internet access to connect to the ThreatConnect API server. In these cases, a proxy server can be used to provide the required connectivity. To configure the Python SDK to use a proxy, call the set_proxies() method, providing the proxy-server IP address and port number as parameters. If the proxy server requires authentication, also provide the proxy user and proxy password as parameters.

Advanced Filtering

A list of Filters can also be retrieved by using the filter1.filters property:

owner = 'Example Community'

filter1 = adversary.add_filter()
filter1.add_owner(owner)
filter1.add_tag('Nation State')

print(filter1)

The Python SDK provides a powerful filtering system. When possible, it allows the user to set API Filters that limit the results returned from the API. If further filtering is required, there are Post Filters that allow the user to further refine the result set. The API Filters in a single Filter object will OR the results together, while the Post Filter will AND the results.

Printing Filter Objects

After creating a Filter object, the object can be printed, which will display the number of Request objects created, as well as the supported API Filters and Post Filters. A list of Filters can also be retrieved by using the filter1.filters property.

filter1.filters Resulting Output

Filter Object
Filter Properties
Operator	FilterSetOperator.AND
Request Objects	1
Owners
Owner	Example Community
Filters
Filter	api filter by tag “Nation State”
API Filters
Filter	`add\_adversary\_id`
Filter	`add\_email\_id`
Filter	`add\_document\_id`
Filter	`add\_id`
Filter	`add\_incident\_id`
Filter	`add\_indicator`
Filter	`add\_security\_label`
Filter	`add\_signature\_id`
Filter	`add\_threat\_id`
Filter	`add\_tag`
Filter	`add\_victim\_id`
Post Filters
Filter	`add\_pf\_name`
Filter	`add\_pf\_date\_added`

Filter Object Basics

Python SDK Filter Object Basics example:

filter1 = adversary.add_filter()
filter1.add_indicator('10.20.30.40')
filter1.add_victim_id(10)
filter1.add_tag('Nation State')

Python SDK Post Filter Basics example:

from threatconnect.Config.FilterOperator import FilterOperator

filter1 = adversary.add_filter()
filter1.add_pf_name('Bad Guy')
filter1.add_pf_date_added('2015-06-18T20:21:45-05:00', FilterOperator.GE)

As mentioned above, an API Filter will join the results. In the example, the API results will contain any Adversary that has an Association with the Indicator 10.20.30.40, OR an Association with the Victim with an ID of 10, OR has the Tag of Nation State.

As mentioned above, the Post Filters will intersect the results. In the example, the API results will only contain Adversaries that have the name “Bad Guy” AND have a date added of >= 2015-06-18T20:21:45-05:00.

Owner API Filter

The Owner API Filter is a special Filter that is applied to all other API Filters in the same Filter Object. This is due to the fact that the API supports adding the Owner as a query String. See the formatted URI examples below.

Python SDK formatted URI examples:

/v2/indicators/address/10.20.30.40?owner=Example+Community

/v2/groups/adversaries/5/indicators?owner=Example+Community

Indicator-Type Filter

An Indicator Filter object supports passing an optional IndicatorType enum argument to the add_filter method. This will filter all results in the Filter object to the Indicator Type specified.

Supported Indicator Types
ADDRESSES
EMAIL_ADDRESSES
FILES
HOSTS
URLS

Python SDK example filtering on supported Indicator Types:

from threatconnect.Config.IndicatorType import IndicatorType

filter1 = indicators.add_filter(IndicatorType.ADDRESSES)
filter1 = indicators.add_filter(IndicatorType.EMAIL_ADDRESSES)
filter1 = indicators.add_filter(IndicatorType.FILES)
filter1 = indicators.add_filter(IndicatorType.HOSTS)
filter1 = indicators.add_filter(IndicatorType.URLS)

Modified Since API Filter

Python SDK Modified Since API Filter:

from datetime import datetime

modified_since = (datetime.isoformat(datetime(2015, 6, 17))) + 'Z'
indicators.set_modified_since(modified_since)

The Modified Since Filter applies to the entire Indicators Container but can only be used on base Indicator searches (e.g., /v2/indicators). If a Filter on modified since is required on a different Indicator search, there is a Post Filter for modified since that works on all Indicator result sets.

Multiple Filter Objects

Python SDK Multiple Filter Objects example:

from threatconnect.Config.FilterOperator import FilterSetOperator
from threatconnect.Config.IndicatorType import IndicatorType

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

owner = 'Example Community'
indicators = tc.indicators()

try:
    filter1 = indicators.add_filter()
    filter1.add_owner(owner)
    filter1.add_security_label('TLP Red')
except AttributeError as e:
    print(e)
    sys.exit(1)

try:
    filter2 = indicators.add_filter()
    filter2.add_owner(owner)
    filter2.add_filter_operator(FilterSetOperator.AND)
    filter2.add_threat_id(38)
except AttributeError as e:
    print(e)
    sys.exit(1)

try:
    filter3 = indicators.add_filter(IndicatorType.ADDRESSES)
    filter3.add_owner(owner)
    filter3.add_filter_operator(FilterSetOperator.OR)
    filter3.add_tag('EXAMPLE')
except AttributeError as e:
    print(e)
    sys.exit(1)

# add code here

The Python SDK supports adding multiple Filter objects to a Resource Container. A filter_operator allows a user to configure the results sets of the separate Filter objects to be JOINED or INTERSECTED. No filter_operator is required on the first Filter object added. Each subsequent Filter object can be joined (FilterSetOperator.OR) or intersected (FilterSetOperator.AND).

Manual API Calls

The Python SDK supports a manual way to access the API by allowing the creation of a RequestObject() and submitting these objects to the api_request() method. The returned result will be a Python Requests object containing the HTTP Status Code, Response Headers, and API Results.

Retrieving Indicators

The example below demonstrates how to create a RequestObject that will retrieve all Indicators from a specified Owner:

import json

from threatconnect.RequestObject import RequestObject

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

owner = 'Example Community'

# instantiate Request Object
ro = RequestObject()

# set http method for Request Object
ro.set_http_method('GET')

# set the owner
ro.set_owner(owner)  # OPTIONAL

# set the Owner-Allowed flag to specify whether or not this API call supports owners
ro.set_owner_allowed(True)

# set the Pagination flag to specify whether or not this API call supports pagination
ro.set_resource_pagination(True)

# set the URI (uniform resource identifier) for the request
ro.set_request_uri('/v2/indicators')

# trigger the request and store the response as results
results = tc.api_request(ro)
if results.headers['content-type'] == 'application/json':
    data = results.json()
    print(json.dumps(data, indent=4))

Downloading Document Contents

The example below demonstrates how to create a RequestObject that will retrieve the contents of a document stored as a Document Resource in ThreatConnect.

from threatconnect.RequestObject import RequestObject

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

owner = 'Example Community'

# instantiate Request Object
ro = RequestObject()

# set http method for Request Object
ro.set_http_method('GET')

# set the owner
ro.set_owner(owner)  # OPTIONAL

# set the Owner-Allowed flag to specify whether or not this API call supports owners
ro.set_owner_allowed(True)

# set the Pagination flag to specify whether or not this API call supports pagination
ro.set_resource_pagination(False)

# set the URI (uniform resource identifier) for the request
ro.set_request_uri('/v2/groups/documents/19/download')

# trigger the request and store the response as results
results = tc.api_request(ro)
if results.headers['content-type'] == 'application/octet-stream':
    file_contents = results.content
    # print the Document's content
    print(file_contents)

Creating and Uploading Documents

The example below demonstrates how to create a RequestObject that will create a Document Resource in ThreatConnect and upload content into this Resource.

import json

from threatconnect.RequestObject import RequestObject

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

owner = 'Example Community'

# instantiate Request Object
ro = RequestObject()

# set http method for Request Object
ro.set_http_method('POST')

# set the body of the request
body = {'name': 'Raw Upload Example', 'fileName': 'raw_example.txt'}
ro.set_body(json.dumps(body))

# set the content type of the request
ro.set_content_type('application/json')

# set the owner
ro.set_owner(owner)  # OPTIONAL

# set the Owner-Allowed flag to specify whether or not this API call supports owners
ro.set_owner_allowed(True)

# set the Pagination flag to specify whether or not this API call supports pagination
ro.set_resource_pagination(False)

# set the URI (uniform resource identifier) for the request
ro.set_request_uri('/v2/groups/documents')

print(ro)

# trigger the request and store the response as results
results = tc.api_request(ro)
if results.headers['content-type'] == 'application/json':
    data = results.json()
    print(json.dumps(data, indent=4))

    # get the ID of the created document
    document_id = data['data']['document']['id']

    # create another Request Object for uploading the document contents
    ro = RequestObject()
    ro.set_http_method('POST')

    # define the Request's body (this is the content that will be uploaded into the Document Resource in ThreatConnect)
    body = 'Raw upload example file Contents.'
    ro.set_body(body)

    ro.set_content_type('application/octet-stream')
    ro.set_owner(owner)
    ro.set_owner_allowed(True)
    ro.set_resource_pagination(False)

    # upload the Request's body into the Document Resource in ThreatConnect
    ro.set_request_uri('/v2/groups/documents/{0}/upload'.format(document_id))

    # trigger the request to upload content into the Document Resource
    results = tc.api_request(ro)
    print('Status Code: {0}'.format(results.status_code))

Advanced Outputs Formats

The Python SDK allows for a Resource to be returned in multiple standard formats. The SDK currently supports the following formats:

CEF (Common Event Format)
CSV (Comma-Separated Values)
JSON (JavaScript® Object Notation)
KeyVal (Key Value)
LEEF (Log Event Extended Format)

CEF

Python SDK CEF Code Sample:

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()
owner = 'Example Community'

try:
    filter1 = indicators.add_filter()
    filter1.add_owner(owner)
    filter1.add_tag('Nation State')
except AttributeError as e:
    print(e)
    sys.exit(1)

try:
    # retrieve the Indicators
    indicators.retrieve()
except RuntimeError as e:
    print(e)
    sys.exit(1)

# iterate through the Indicators
for indicator in indicators:
    print(indicator.cef)
    print('')

Python SDK Sample CEF Output:

CEF:0|threatconnect|threatconnect|2|355999|TEST attribute #14|2.0|confidence="14" dateAdded="2015-06-21T10:40:33-05:00" dnsActive="None" hostName="www.badguy_014.com" lastModified="2015-06-21T10:40:33-05:00" ownerName="Example Community" type="None" weblink="https://tc.sumx.us/auth/indicators/details/host.xhtml?host\=www.badguy_014.com&owner\=Example+Community" whoisActive="None"

The Python SDK provides the cef methods to output data structured in CEF, whose output is only supported on Indicators. The CEF-formatted data maps the ThreatConnect Resource properties to the standard fields, when possible, and then uses the extension feature to store non-standard properties.

CSV

Python SDK CSV Code Sample:

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()
owner = 'Example Community'

try:
    filter1 = indicators.add_filter()
    filter1.add_owner(owner)
    filter1.add_tag('Nation State')
except AttributeError as e:
    print(e)
    sys.exit(1)

try:
    # retrieve the Indicators
    indicators.retrieve()
except RuntimeError as e:
    print(e)
    sys.exit(1)

for indicator in indicators:
    print(indicator.csv_header)
    print(indicator.csv)
    print('')

Python SDK Sample CSV Output:

confidence,dateAdded,description,id,indicator,lastModified,ownerName,rating,type,weblink
14,2015-06-21T10:40:33-05:00,TEST attribute #14,355999,www.badguy.com,2015-06-21T10:40:33-05:00,Example Community,1.0,null,https://app.threatconnect.com/auth/indicators/details/host.xhtml?host=www.badguy.com&owner=Example+Community

The Python SDK provides the csv and csv_header methods for CSV output, which are supported on Indicators as well as Group Resources (e.g., Adversaries, Documents, Emails, Incidents, Signatures and Threats)

The csv_header method should normally be called once per result set.

JSON

Python SDK JSON Code Sample:

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()
owner = 'Example Community'

try:
    filter1 = indicators.add_filter()
    filter1.add_owner(owner)
    filter1.add_tag('Nation State')
except AttributeError as e:
    print(e)
    sys.exit(1)

try:
    # retrieve the Indicators
    indicators.retrieve()
except RuntimeError as e:
    print(e)
    sys.exit(1)

# iterate through the Indicators
for indicator in indicators:
    print(indicator.json)
    print('')

Python SDK Sample JSON Output:

{
    "confidence": 14,
    "dateAdded": "2015-06-21T10:40:33-05:00",
    "description": "TEST attribute #14",
    "dnsActive": null,
    "hostName": "www.badguy_014.com",
    "id": 355999,
    "lastModified": "2015-06-21T10:40:33-05:00",
    "ownerName": "Example Community",
    "rating": 1.0,
    "type": null,
    "weblink": "https://tc.sumx.us/auth/indicators/details/host.xhtml?host=www.badguy_014.com&owner=Example+Community",
    "whoisActive": null
}

The Python SDK provides the json method for output in JSON, are supported on Indicators as well as Group Resources (e.g., Adversaries, Documents, Emails, Incidents, Signatures and Threats)

The fields in the output depend on the type of Resource that has been requested.

Key Value

Python SDK Key Value Code Sample:

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()
owner = 'Example Community'

try:
    filter1 = indicators.add_filter()
    filter1.add_owner(owner)
    filter1.add_tag('Nation State')
except AttributeError as e:
    print(e)
    sys.exit(1)

try:
    # retrieve the Indicators
    indicators.retrieve()
except RuntimeError as e:
    print(e)
    sys.exit(1)

# iterate through the Indicators
for indicator in indicators:
    print(indicator.keyval)

Sample Key/Value Output:

confidence="14" dateAdded="2015-06-21T10:40:33-05:00" description="TEST attribute #14" dnsActive="None" hostName="www.badguy_014.com" id="355999" lastModified="2015-06-21T10:40:33-05:00" ownerName="Example Community" rating="1.0" type="None" weblink="https://tc.sumx.us/auth/indicators/details/host.xhtml?host=www.badguy_014.com&owner=Example+Community" whoisActive="None"

The Python SDK provides the keyval method for output in the Key Value format, whose output is supported on Indicators as well as Group Resources (e.g., Adversaries, Documents, Emails, Incidents, Signatures and Threats)

The fields in the output depend on the type of Resource that has been requested.

LEEF

Python SDK LEEF Code Sample:

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()
owner = 'Example Community'

try:
    filter1 = indicators.add_filter()
    filter1.add_owner(owner)
    filter1.add_tag('Nation State')
except AttributeError as e:
    print(e)
    sys.exit(1)

try:
    # retrieve the Indicators
    indicators.retrieve()
except RuntimeError as e:
    print(e)
    sys.exit(1)

# iterate through the Indicators
for indicator in indicators:
    print(indicator.leef)
    print('')

Python SDK Sample LEEF Output:

LEEF:0|threatconnect|threatconnect|2|355999|confidence="14" devTime="2015-06-21T10:40:33-05:00" description="TEST attribute #14" dnsActive="None" hostName="www.badguy_014.com" id="355999" lastModified="2015-06-21T10:40:33-05:00" ownerName="Example Community" severity="1.0" type="None" weblink="https://tc.sumx.us/auth/indicators/details/host.xhtml?host=www.badguy_014.com&owner=Example+Community" whoisActive="None"

The Python SDK provides the leef method to output data structured in LEEF, whose output is only supported on Indicators. The LEEF-formatted data maps the ThreatConnect Resource properties to the standard fields, when possible, and then uses the custom attribute feature to store non-standard properties.

Indicator Type Override

The add() method on the tc.indicators() object allows the user to bypass the automatic Indicator identification and validation check by specifying the IndicatorType:

from threatconnect.Config.IndicatorType import IndicatorType

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()
owner = 'Example Community'

indicator = indicators.add('<indicator>', owner, IndicatorType.ADDRESSES)

Regex Overrides

Python SDK Regex Code Sample

import re

from threatconnect.Config.IndicatorType import IndicatorType

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()

#
# override FILES Regex
#
md5_re = re.compile(r'^([a-fA-F\d]{32})$')
sha1_re = re.compile(r'^([a-fA-F\d]{40})$')
sha256_re = re.compile(r'^([a-fA-F\d]{64})$')
tc.set_indicator_regex(IndicatorType.FILES, [md5_re, sha1_re, sha256_re])

#
# override ADDRESSES Regex
#
ipv4_regex = re.compile('(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}' +
                         '(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)')
ipv6_regex = re.compile('(S*([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}S*|S*(' +
                         '[0-9a-fA-F]{1,4}:){1,7}:S*|S*([0-9a-fA-F]{1,4}:)' +
                         '{1,6}:[0-9a-fA-F]{1,4}S*|S*([0-9a-fA-F]{1,4}:)' +
                         '{1,5}(:[0-9a-fA-F]{1,4}){1,2}S*|S*([0-9a-fA-F]' +
                         '{1,4}:){1,4}(:[0-9a-fA-F]{1,4}){1,3}S*|S*(' +
                         '[0-9a-fA-F]{1,4}:){1,3}(:[0-9a-fA-F]{1,4}){1,4}S*' +
                         '|S*([0-9a-fA-F]{1,4}:){1,2}(:[0-9a-fA-F]{1,4})' +
                         '{1,5}S*|S*[0-9a-fA-F]{1,4}:((:[0-9a-fA-F]{1,4})' +
                         '{1,6})S*|S*:((:[0-9a-fA-F]{1,4}){1,7}|:)S*|::(ffff' +
                         '(:0{1,4}){0,1}:){0,1}((25[0-5]|(2[0-4]|1{0,1}' +
                         '[0-9]){0,1}[0-9]).){3,3}(25[0-5]|(2[0-4]|1{0,1}[' +
                         '0-9]){0,1}[0-9])|([0-9a-fA-F]{1,4}:){1,4}:((25[' +
                         '0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]).){3,3}(25[' +
                         '0-5]|(2[0-4]|1{0,1}[0-9]){0,1}[0-9]))')
tc.set_indicator_regex(IndicatorType.ADDRESSES, [ipv4_regex, ipv6_regex])

#
# override HOSTS Regex
#
host_re = re.compile(r'\b((?:(?!-)[a-zA-Z0-9-]{1,63}(?<!-)\.)+(?i)(?!exe|php|dll|doc' \
    '|docx|txt|rtf|odt|xls|xlsx|ppt|pptx|bin|pcap|ioc|pdf|mdb|asp|html|xml|jpg|gif$|png' \
    '|lnk|log|vbs|lco|bat|shell|quit|pdb|vbp|bdoda|bsspx|save|cpl|wav|tmp|close|ico|ini' \
    '|sleep|run|dat$|scr|jar|jxr|apt|w32|css|js|xpi|class|apk|rar|zip|hlp|cpp|crl' \
    '|cfg|cer|plg|lxdns|cgi|xn$)(?:xn--[a-zA-Z0-9]{2,22}|[a-zA-Z]{2,13}))(?:\s|$)')
tc.set_indicator_regex(IndicatorType.HOSTS, host_re)

indicator = indicators.add('new.domain.tld', owner)
indicator.set_confidence(50)
indicator.set_rating('2.0')

try:
    # commit the Indicator
    indicator.commit()
except RuntimeError as e:
    print('Error: {0!s}'.format(e))
    sys.exit(1)

The Python SDK provides the set_indicator_regex method which allows a user to override the baked-in Regular Expressions (Regexes) in the SDK with user defined compiled Regexes. The method takes an IndicatorType enum and either a single compiled Regex or a list of Regexes. If a list is provided each Regex will be checked for a match for that Indicator Type.

Reporting

Stats Reporting

The tc.report.stats properties method provides an overview of the script results:

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()
owner = 'Example Community'

filter1 = indicators.add_filter()
filter1.add_owner(owner)
filter1.add_tag('Nation State')

try:
    # retrieve the Indicators
    indicators.retrieve()
except RuntimeError as e:
    print('Error: {0}'.format(e))
    sys.exit(1)
else:
    print(tc.report.stats)

Sample Report-Statistics Output:

_Stats_

API Stats
  API Calls                    32
  Unfiltered Results           3
  Filtered Results             3

Filters
  API Filters                  1
  Post Filters                 0
  Total Filters                1

HTTP Methods
  PUT                          2
  POST                         11
  DELETE                       11
  GET                          8

Status Codes
  200                          21
  201                          11

Performance Stats
  Request Time           0:00:03.021702
  Processing Time        0:00:00.014082
  Run Time               0:00:03.035795

The Python SDK includes a reporting feature that provides a number of methods for reporting on the execution status of a script that uses the SDK.

Enabling Reporting

The basic data collection of the Reporting feature is always enabled, but the report-entry collection feature is disabled by default. To enable the report-entry collection feature, use the tc.report_enable() method. To disable reporting, use the tc.report_disable() method.

Statistics

The tc.report.stats properties method provides an overview of the script results.

Failed Reports

Python SDK failed reports example:

# iterate through the failures
for fail in tc.report.failures:
    print(fail)
    print('')

Sample Failed-Report Output:

_Report Entry_

Properties
Status Code
: 404
Fail Msg
: {"status":"Failure","message":"The requested resource was not found"}
Description
: api filter by incident id 708996
Resource Type
: ResourceType.ADVERSARIES

HTTP Settings
  HTTP Method
  GET
  Request URI
  /v2/groups/incidents/708996/groups/adversaries
  Request URL
  https://tc.sumx.us/api/v2/groups/incidents/708996/groups/adversaries?resultStart=0&resultLimit=500&createActivityLog=false
  Content Type                 None
  Body                         None

Payload
  Payload
  {'resultStart': 0, 'resultLimit': 500, 'createActivityLog': 'false'}

All API requests and Post Filters are stored as a report entry in the Reports object. Any request that does not receive a status code of 200, 201 or 202, is stored as a failed-report entry and can be retrieved with the tc.report.failures property method. This feature helps debug issues when receiving failures while communicating with the API.

Other Reporting Features

API Calls

The number of API calls can be retrieved using the tc.report.api_calls property method of the Report object.

Runtime

The script execution time can be retrieved using the tc.report.runtime property method of the Report object. This method can be called anytime during the script execution to get the current runtime and at the end of the script to get the total runtime.

Request Time

The time spent on API requests can be retrieved using the tc.report.request_time property method of the Report object.

Report Entries

All report entries can be accessed via the Report generator. By iterating over tc.report, each individual report entry will be returned. These report entries can be printed and the individual properties can be accessed.

Gotchas

This section details some things to be aware of when using the Python SDK for advanced use-cases.

Order is Important when Adding Attributes

If you are adding attributes to an indicator using this SDK, the order in which the attributes are added can be important. This is true if one of the attributes may be improperly formatted, thus causing an API error.

To illustrate this, consider the following code:

# replace the line below with the standard, TC script heading described here:
# https://docs.threatconnect.com/en/latest/python/quick_start.html#standard-script-heading
...

tc = ThreatConnect(api_access_id, api_secret_key, api_default_org, api_base_url)

# instantiate Indicators object
indicators = tc.indicators()

owner = 'Example Community'

# create a new file indicator
indicator = indicators.add('a'*32, owner)

# add a Description attribute
indicator.add_attribute('Description', 'Test description')

# add an ssdeep Hash attribute
indicator.add_attribute('ssdeep Hash', '!!MALFORMED SSDEEP HASH!!')

# add a Source attribute
indicator.add_attribute('Source', 'Test source')

# set the confidence rating for the indicator
indicator.set_confidence(75)

indicator.commit()

We want to create a File Indicator, add three attributes (description, ssdeep hash, and source), and set the confidence rating. When indicator.commit() is called, it will follow these steps (the important sections are in bold):

Create the indicator
Add a description attribute
Fail while trying to add the ssdeep Hash attribute
Will not add a source attribute
Set the confidence rating

The key point is that any attributes created after the creation of another attribute has failed will not be created. Thus, if you have an attribute that may be invalid, you should add other attributes first. Other operations like setting the confidence and threat ratings and adding tags will work properly after the creation of an attribute has failed.