Bin Module

bin.bin

TcEx Framework Bin Command Base Module.

class tcex.bin.bin.Bin(_args)

Bases: object

Base Class for ThreatConnect command line tools.

Parameters:_args (namespace) – The argparser args Namespace.

db_conn

Create a temporary in memory DB and return the connection.

db_create_table(table_name, columns)

Create a temporary DB table.

Parameters:

• table_name (str) – The name of the table.
• columns (list) – List of columns to add to the DB.

db_insert_record(table_name, columns)

Insert records into DB.

Parameters:

• table_name (str) – The name of the table.
• columns (list) – List of columns for insert statement.

db_update_record(table_name, column, value)

Insert records into DB.

Parameters:

• table_name (str) – The name of the table.
• column (str) – The column name in which the value is to be updated.
• value (str) – The value to update in the column.

gen_permutations(index=0, args=None)

Iterate recursively over layout.json parameter names to build permutations.

Note

Permutations are for layout.json based playbook Apps.

Parameters:

• index (int, optional) – The current index position in the layout names list.
• args (list, optional) – Defaults to None. The current list of args.

static handle_error(err, halt=True)

Print errors message and optionally exit.

Parameters:

• err (str) – The error message to print.
• halt (bool, optional) – Defaults to True. If True the script will exit.

permutations()

Process layout.json names/display to get all permutations of args.

print_permutations()

Print all valid permutations.

profile_settings_args_layout_json(required)

Return args based on layout.json and conditional rendering.

Parameters:required (bool) – If True only required args will be returned. Returns:Dictionary of required or optional App args. Return type:dict

redis

Return instance of Redis.

tcex_json

Return tcex.json file contents.

static update_system_path()

Update the system path to ensure project modules and dependencies can be found.

validate_layout_display(table, display_condition)

Check to see if the display condition passes.

Parameters:

• table (str) – The name of the DB table which hold the App data.
• display_condition (str) – The “where” clause of the DB SQL statement.

Returns:

True if the row count is greater than 0.

Return type:

bool

bin.init

TcEx App Init.

class tcex.bin.init.Init(_args)

Bases: tcex.bin.bin.Bin

Install required modules for ThreatConnect Job or Playbook App.

Parameters:_args (namespace) – The argparser args Namespace.

check_empty_app_dir()

Check to see if the directory in which the app is going to be created is empty.

download_file(remote_filename, local_filename=None)

Download file from github.

Parameters:

• remote_filename (str) – The name of the file as defined in git repository.
• local_filename (str, optional) – Defaults to None. The name of the file as it should be be written to local filesystem.

static update_install_json()

Update the install.json configuration file if exists.

update_tcex_json()

Update the tcex.json configuration file if exists.

bin.lib

TcEx Library Builder.

class tcex.bin.lib.Lib(_args)

Bases: tcex.bin.bin.Bin

Install Required Modules for App.

Parameters:_args (namespace) – The argparser args Namespace.

_build_command(python_executable, lib_dir_fq, proxy_enabled)

Build the pip command for installing dependencies.

Parameters:

• python_executable (str) – The fully qualified path of the Python executable.
• lib_dir_fq (str) – The fully qualified path of the lib directory.

Returns:

The Python pip command with all required args.

Return type:

list

_configure_proxy()

Configure proxy settings using environment variables.

_create_lib_latest()

Create the lib_latest symlink for App Builder.

_create_temp_requirements()

Create a temporary requirements.txt.

This allows testing again a git branch instead of pulling from pypi.

install_libs()

Install Required Libraries using pip.

bin.package

TcEx Framework Package Module.

class tcex.bin.package.Package(_args)

Bases: tcex.bin.bin.Bin

Package ThreatConnect Job or Playbook App for deployment.

This method will package the app for deployment to ThreatConnect. Validation of the install.json file or files will be automatically run before packaging the app.

Parameters:_args (namespace) – The argparser args Namespace.

_update_install_json(install_json)

Update install.json file.

Parameters:install_json (dict) – The contents of the install.json file. Returns:

The contents of the install.json file and boolean value that is True if

an update was made.

Return type:dict, bool

_write_install_json(filename, install_json)

Write install.json file.

Some projects have bundles App with multiple install.json files. Typically these files are prefixed with the App name (e.g., MyApp.install.json).

Parameters:

• filename (str) – The install.json file name.
• install_json (dict) – The contents of the install.json file.

app_name(install_json_name)

Return the app package name without version.

1. Use the prefix on the install.json file (bundled Apps).
2. Use the app_name field from the tcex.json file.
3. Use the app directory name. This option should not be used.

app_version(ij)

Return the app version “v1”.

1. Use app_version value from tcex.json if available. Typicall version is major only (e.g., v1), but from older Apps it could have minor version.
2. Use major version from programVersion field in install.json if available.
3. Default to ‘1.0.0’ updated to major version only (‘v1’).

bundle(bundle_name)

Bundle multiple Job or Playbook Apps into a single zip file.

Parameters:bundle_name (str) – The output name of the bundle zip file.

bundle_apps(bundle_name, bundle_apps)

Bundle multiple Job or Playbook Apps (.tcx files) into a single zip file.

Parameters:

• bundle_name (str) – The output name of the bundle zip file.
• bundle_apps (list) – A list of Apps to include in the bundle.

commit_hash

Return the current commit hash if available.

This is not a required task so best effort is fine. In other words this is not guaranteed to work 100% of the time.

package()

Build the App package for deployment to ThreatConnect Exchange.

print_json()

Print JSON output containing results of the package command.

print_results()

Print results of the package command.

zip_file(app_path, app_name, tmp_path)

Zip the App with tcex extension.

Parameters:

• app_path (str) – The path of the current project.
• app_name (str) – The name of the App.
• tmp_path (str) – The temp output path for the zip.

bin.test

TcEx Framework Profile Generation Module.

class tcex.bin.test.CustomTemplates(base_dir, branch='master')

Custom method Template Class

feature_template

Return the feature template file for custom methods

parent_template

Return the parent template file for custom methods

render_templates(feature, app_type)

Render the templates and write to disk conditionally.

class tcex.bin.test.Profiles(profile_dir)

Profile Class

Raises:

• NotImplementedError – The delete method is not currently implemented.
• NotImplementedError – The update method is not currently implemented.

add(profile_name, profile_data, sort_keys=True)

Add a profile.

delete(profile_name)

Delete an existing profile.

update(profile_name)

Update an existing profile.

class tcex.bin.test.Test(_args)

Bases: tcex.bin.bin.Bin

Create profiles for ThreatConnect Job or Playbook App.

Parameters:_args (namespace) – The argparser args Namespace.

add_profile()

Add the desired profile

static add_profile_staging(staging_files)

Get existing staging data.

create_dirs()

Create tcex.d directory and sub directories.

create_dirs_init()

Create the __init__.py file under dir.

download_conftest()

Download conftest.py file from github.

download_file(remote_filename)

Download file from github.

Parameters:remote_filename (str) – The name of the file as defined in git repository.

download_profile()

Download conftest.py file from github.

generate_custom_files()

Generate the custom.py and custom_feature.py files.

generate_test_profile_file()

Generate the test_profiles.py file.

generate_validation_files()

Generate the validation.py and validation_feature.py files.

output_variables

Return playbook output variables

permutation_file_exists()

Check to see if the permutations file exists

permutations_file

Return permutations fully qualified filename.

profiles_dir

Return profile fully qualified filename.

test_file

Return generate test filename.

validation_base_file

Return validations fully qualified filename.

validation_file

Return validations fully qualified filename.

class tcex.bin.test.TestProfileTemplates(base_dir, branch='master')

TestProfile Template Class

render_template(feature, app_type)

Render the templates and write to disk conditionally.

test_profiles_template

Return template file

class tcex.bin.test.ValidationTemplates(base_dir, branch='master')

Validation Template Class

Raises:

• NotImplementedError – The delete method is not currently implemented.
• NotImplementedError – The update method is not currently implemented.

_method_name(variable)

Convert variable name to a valid method name.

Parameters:variable (string) – The variable name to convert. Returns:Method name Return type:(str)

custom_template

Return template file

feature_template

Return template file

output_data(output_variables)

Return formatted output data.

#App:9876:http.content!Binary

parent_template

Return template file

render_templates(feature, app_type, output_variables)

Render the templates and write to disk conditionally.

bin.validate

TcEx Framework Validate Module.

class tcex.bin.validate.Validate(_args)

Bases: tcex.bin.bin.Bin

Validate syntax, imports, and schemas.

• Python and JSON file syntax
• Python import modules
• install.json schema
• layout.json schema

Parameters:_args (namespace) – The argparser args Namespace.

static check_import_stdlib(module)

Check if module is in Python stdlib.

Parameters:module (str) – The name of the module to check. Returns:Returns True if the module is in the stdlib or template. Return type:bool

static check_imported(module)

Check whether the provide module can be imported (package installed). :param module: The name of the module to check availability. :type module: str

Returns:True if the module can be imported, False otherwise. Return type:bool

check_imports()

Check the projects top level directory for missing imports.

This method will check only files ending in .py and does not handle imports validation for sub-directories.

check_install_json()

Check all install.json files for valid schema.

check_layout_json()

Check all layout.json files for valid schema.

check_layout_params()

Check that the layout.json is consistent with install.json.

The layout.json files references the params.name from the install.json file. The method will validate that no reference appear for inputs in install.json that don’t exist.

check_syntax(app_path=None)

Run syntax on each “.py” and “.json” file.

Parameters:app_path (str, optional) – Defaults to None. The path of Python files.

install_json_schema

Load install.json schema file.

interactive()

Run in interactive mode.

layout_json_schema

Load layout.json schema file.

print_json()

Print JSON output.

print_results()

Print results.

static status_color(status)

Return the appropriate status color.

static status_value(status)

Return the appropriate status color.