Module: Threat Intelligence

The ThreatConnect TcEx Framework provides the ThreatIntelligence module which provides the following functionality:

  • Create - Groups, Indicators, Tasks, and Victims.

  • Delete - Groups, Indicators, Tasks, and Victims.

  • Get - Groups, Indicators, Tasks, and Victims.

  • Update - Groups, Indicators, Tasks, and Victims.

  • Get by Tag

  • All Create, Get, and Update methods include TI metadata (e.g., attributes, security labels, and tags).

Single Results

When using the create, delete, get, or update methods for a single object the module returns a Python Request Response Object (https://2.python-requests.org/en/v1.1.0/api/#requests.Response). The Python Requests Response object has the following common properties: status_code, headers, response.text, response.content. To get a Python dict from the response the response.json() method can be used.

Multiple Results

When working with multiple objects the modules yields the ThreatConnect object entity as a Python dict. The ThreatConnect entity dict format varies depending on the type and matches the data response from the API.

Groups

Get Groups by Type

To retrieve all Groups of a specific type from the ThreatConnect REST API, the group_type can be provided to the group() method. Once the Group object is initialized, the many() method allows pagination over all Groups.

The example below retrieves all Groups of type Adversary.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2groups = self.tcex.ti.group(group_type='Adversary', owner='MyOrg')
3for group in groups.many(params=parameters):
4    self.tcex.log.debug('group: {}'.format(group))

Get Groups by Tag

To retrieve all Groups with a specific Tag from the ThreatConnect REST API, the Tag name can be provided to the tag() method. Once the Tag object is initialized, the groups() method retrieves all Groups with that Tag and allows pagination over them.

The example below retrieves all Groups with a group of Crimeware.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2tag = self.tcex.ti.tag('Crimeware')
3for group in tag.groups(params=parameters):
4    self.tcex.log.debug('group: {}'.format(group))

Get Groups by Filter

To retrieve all Groups using a Filter from the ThreatConnect REST API, the Group Filters can be provided to the many() method. The calling many() method allows pagination over all Groups matching the provided Filter(s). The filters added should be the same filter parameters as the REST API expects.

The example below retrieves all Groups where the name equals my_name_filter.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2groups = self.tcex.ti.group(owner='MyOrg')
3filters = self.tcex.ti.filters()
4filters.add_filter('name', '=', 'my_name_filter')
5for group in groups.many(filters=filters, params=parameters):
6    self.tcex.log.debug('group: {}'.format(group))

Get Group by ID

To retrieve a Group from the ThreatConnect REST API, the Group’s unique ID is required. Once the Group object is initialized, the single method will return the Group data from the API.

The example below retrieves a single Adversary with ID 416, if it exists.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2ti = self.tcex.ti.group(group_type='Adversary', owner='MyOrg', unique_id=416)
3response = ti.single(params=parameters)
4group = response.json().get("data", {})

Get Group Metadata

There are six methods that retrieve metadata for a Group: group_associations(), indicator_associations(), victim_asset_associations(), attributes(), tags(), and labels(). Attributes, Labels, and Tags can also be retrieved in the initial API call by using the include query parameters.

 1ti = self.tcex.ti.group(group_type='Adversary', owner='MyOrg', unique_id=416)
 2
 3# get group associations
 4for association in ti.group_associations():
 5    self.tcex.log.debug('association: {}'.format(association))
 6
 7# get indicator associations
 8for association in ti.indicator_associations():
 9    self.tcex.log.debug('association: {}'.format(association))
10
11# get victim asset associations
12for association in ti.victim_asset_associations():
13    self.tcex.log.debug('association: {}'.format(association))
14
15# get attributes
16for attribute in ti.attributes():
17    self.tcex.log.debug('attribute: {}'.format(attribute))
18
19# get tags
20for tag in ti.tags():
21    self.tcex.log.debug('tag: {}'.format(tag))
22
23# get security labels
24for label in ti.labels():
25    self.tcex.log.debug('label: {}'.format(label))

Create a Group

Creating a Group and adding Associations and metadata constitute separate API calls, and each have their own Response object.

 1ti = self.tcex.ti.group(group_type='Campaign', name='camp-3', owner='MyOrg', first_seen='2019-04-02')
 2response = ti.create()
 3
 4# add associations
 5group_assoc = self.tcex.ti.group(group_type='Adversary', owner='MyOrg', unique_id=417)
 6response = ti.add_association(target=group_assoc)
 7
 8# add attributes
 9response = ti.add_attribute(attribute_type='Description', attribute_value='Example Description.')
10
11# add security label
12response = ti.add_label(label='TLP:GREEN')
13
14# add tag
15response = ti.add_tag(name='Crimeware')

To create a Group, you will need to provide a keyword argument with the each of the required group fields for the type of Group you would like to create.

Group Creation Examples

Here are examples creating each Group type:

Adversary:

1group_object = self.tcex.ti.group(group_type='Adversary', name='Adversary 1', owner='MyOrg')
2response = group_object.create()

Campaign:

1group_object = self.tcex.ti.group(group_type='Campaign', name='Campaign 1', owner='MyOrg')
2response = group_object.create()

Document:

1group_object = self.tcex.ti.group(group_type='Document', name='Document 1', owner='MyOrg', file_name='test.txt'))
2response = group_object.create()
3group_object.file_content('This is the content of the document')

Email:

1group_object = self.tcex.ti.group(group_type='Email', name='Email 1', owner='MyOrg', subject='Foo', header='This an email header', body='This is an email body')
2response = group_object.create()

Event:

1group_object = self.tcex.ti.group(group_type='Event', name='Event 1', owner='MyOrg')
2response = group_object.create()

Incident

1group_object = self.tcex.ti.group(group_type='Incident', name='Incident 1', owner='MyOrg')
2response = group_object.create()

Intrusion Set:

1group_object = self.tcex.ti.group(group_type='Intrusion Set' name='Intrusion Set 1', owner='MyOrg')
2response = group_object.create()

Report:

1group_object = self.tcex.ti.group(group_type='Report', name='Report 1', owner='MyOrg', file_name='report.html')
2response = group_object.create()
3group_object.file_content('<h1>New report</h1>\nReport contents here...')

Signature:

 1signature_text = '''rule silent_banker : banker
 2{
 3    meta:
 4        description = "This is just an example"
 5        threat_level = 3
 6        in_the_wild = true
 7    strings:
 8        $a = {6A 40 68 00 30 00 00 6A 14 8D 91}
 9        $b = {8D 4D B0 2B C1 83 C0 27 99 6A 4E 59 F7 F9}
10        $c = "UVODFRYSIHLNWPEJXQZAKCBGMT"
11    condition:
12        $a or $b or $c
13}'''
14group_object = self.tcex.ti.group(group_type='Signature', name='s1', owner=OWNER, fileName='sig.yara', fileType='YARA', file_text=signature_text)
15response = group_object.create()

Threat:

1group_object = self.tcex.ti.group(group_type='Threat', name='Threat 1', owner='MyOrg')
2response = group_object.create()

Updating a Group

Updating a Group is similar to creating a Group, with the addition of providing the Group ID. The Group metadata can be updated using the same methods as were used in the Group Create example.

1ti = self.tcex.ti.group(group_type='Campaign', name='camp-3', owner='MyOrg', first_seen='2019-04-02', unique_id=4127)
2response = ti.update()

Delete a Group

Deleting a Group is similar to creating a Group, with the addition of providing the Group ID.

1ti = self.tcex.ti.group(group_type='Campaign', unique_id=4127)
2response = ti.delete()

Indicators

Get Indicator by Type

To retrieve all Indicators of a specific type from the ThreatConnect REST API, the Indicator_type can be provided to the indicator() method. Once the Indicator object is initialized, the many() method allows pagination over all Indicators.

The example below retrieves all Indicators of type Address.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2indicators = self.tcex.ti.indicator(indicator_type='Address', owner='MyOrg')
3for indicator in indicators.many(params=parameters):
4    self.tcex.log.debug('indicator: {}'.format(indicator))

Get Indicators by Tag

To retrieve all Indicators with a specific Tag from the ThreatConnect REST API, the tag name can be provided to the tag() method. Once the Tag object is initialized, the indicators() method retrieves all Indicators with the Tag and allows pagination over them.

The example below retrieves all indicators with a tag of Crimeware.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2tag = self.tcex.ti.tag('Crimeware')
3for indicator in tag.indicators(params=parameters):
4    self.tcex.log.debug('indicator: {}'.format(indicator))

Get Indicators by Filter

To retrieve all Indicators using a Filter from the ThreatConnect REST API, the Indicator Filters can be provided to the many() method. Calling many() method allows pagination over all Indicators matching the provided Filter(s). The filters added should be the same filter parameters as the REST API expects.

The example below retrieves all Indicators where name equals my_name_filter.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2indicators = self.tcex.ti.indicator(owner='MyOrg')
3filters = self.tcex.ti.filters()
4filters.add_filter('summary', '=', 'my_name_filter')
5for indicator in indicators.many(filters=filters, params=parameters):
6    self.tcex.log.debug('indicator: {}'.format(indicator))

Get Indicator by Value

To retrieve an Indicators from the ThreatConnect REST API, the Indicator’s unique Value and Owner are required. Once the Indicator object is initialized, the single method will return the Indicator data from the API.

The example below retrieves a single Address with Value 1.1.1.1, if it exists.

1parameters = {'includes': ['additional', 'attributes', 'labels', 'tags']}
2ti = self.tcex.ti.indicator(indicator_type='Address', owner='MyOrg', unique_id='1.1.1.1')
3response = ti.single(params=parameters)
4indicator = response.json().get("data", {})

Get Indicator Metadata

There are six methods that retrieve metadata for an Indicator: group_associations(), indicator_associations(), attributes(), tags(), and labels(). Attributes, Labels, and Tags can also be retrieved in the initial API call by using the include query parameters.

 1ti = self.tcex.ti.indicator(indicator_type='Address', owner='MyOrg', unique_id='1.1.1.1')
 2
 3# get group associations
 4for association in ti.group_associations():
 5    self.tcex.log.debug('association: {}'.format(association))
 6
 7# get indicator associations
 8for association in ti.indicator_associations():
 9    self.tcex.log.debug('association: {}'.format(association))
10
11# get attributes
12for attribute in ti.attributes():
13    self.tcex.log.debug('attribute: {}'.format(attribute))
14
15# get tags
16for tag in ti.tags():
17    self.tcex.log.debug('tag: {}'.format(tag))
18
19# get security labels
20for label in ti.labels():
21    self.tcex.log.debug('label: {}'.format(label))

Create an Indicator

Creating an Indicator and adding Associations and metadata are all separate API calls, and each have their own Response object.

 1ti = self.tcex.ti.indicator(indicator_type='Host', owner='MyOrg', hostname='example.com')
 2response = ti.create()
 3
 4# add associations
 5group_assoc = self.tcex.ti.group(group_type='Adversary', owner='MyOrg', unique_id=417)
 6response = ti.add_association(target=group_assoc)
 7
 8# add attributes
 9response = ti.add_attribute(attribute_type='Description', attribute_value='Example Description.')
10
11# add security label
12response = ti.add_label(label='TLP:GREEN')
13
14# add tag
15response = ti.add_tag(name='Crimeware')

To create an Indicator, you will need to provide a keyword argument with each of the required indicator fields for the type of Indicator you would like to create.

Indicator Creation Examples

Here are examples creating the five, basic Indicator types:

Address:

1indicator_object = self.tcex.ti.indicator(indicator_type='Address', owner='MyOrg', ip='4.3.2.1')
2response = indicator_object.create()

Email Address:

1indicator_object = self.tcex.ti.indicator(indicator_type='EmailAddress', owner='MyOrg', address='[email protected]')
2response = indicator_object.create()

File:

1# creation args for file indicators - at least one hash type required
2indicator_object = self.tcex.ti.indicator(indicator_type='File', owner='MyOrg', md5='a'*32, sha1="a"*40, sha256="a"*64, size=512)
3response = indicator_object.create()

Host:

1indicator_object = self.tcex.ti.indicator(indicator_type='Host', owner='MyOrg', hostName='example.org')
2response = indicator_object.create()

URL:

1# tcex 2.0.x uses `text` for url keyword, previous versions used `url`
2if tcex.__version__ < "2.0":
3    indicator_object = self.tcex.ti.indicator(indicator_type='URL', owner='MyOrg', url='https://example.org/foo')
4else:
5    indicator_object = self.tcex.ti.indicator(indicator_type='URL', owner='MyOrg', text='https://example.org/foo')
6
7response = indicator_object.create()

Updating an Indicator

Updating an Indicator is similar to creating an Indicator, with the addition of providing the Indicator value. The Indicator metadata can be updated using the same methods as were used in the Indicator Create example.

1ti = self.tcex.ti.indicator(indicator_type='Host', owner='MyOrg', hostname='example.com', dns_active=True, whois_active=True, active=True)
2response = ti.update()

Delete an Indicator

Deleting an indicator is similar to creating an Indicator, with the addition of providing the Indicator value.

1ti = self.tcex.ti.indicator(indicator_type='Address', owner='MyOrg', ip='12.13.14.15')
2response = ti.delete()

Owners

Get Available Owners

To retrieve all Owners available from the ThreatConnect REST API, the owner object can be used. The calling many() method allows pagination over all Owners available.

1ti = self.tcex.ti.owner()
2for owner in ti.many():
3    self.tcex.log.debug('owner: {}'.format(owner))

Get My Owners

To retrieve all Owners that are currently owned from the ThreatConnect REST API, the owner object can be used. The calling mine() method allows pagination over all Owners available.

1ti = self.tcex.ti.owner()
2for owner in ti.mine():
3    self.tcex.log.debug('owner: {}'.format(owner))

Get My Members

To retrieve all Members currently in Owners that are owned from the ThreatConnect REST API, the owner object can be used. The calling members() method allows pagination over all Members available.

1ti = self.tcex.ti.owner()
2for member in ti.members():
3    self.tcex.log.debug('member: {}'.format(member))

Get Metrics

To retrieve all Metrics from the ThreatConnect REST API, the owner object can be used. The calling metrics() method allows pagination over all Metrics available.

1ti = self.tcex.ti.owner()
2for metric in ti.metrics():
3    self.tcex.log.debug('metric: {}'.format(metric))