Command-Line Usage ================== Each tool of ``nbtoolbelt`` is invoked as a subcommand of ``nbtb``: .. code-block:: bash nbtb ... ``nbtb`` handles the following arguments itself: * ``nbtb -version`` to obtain the current version, and exit * ``nbtb -h`` or ``nbtb --help`` to get a usage message, and exit * ``nbtb config`` to show the location of the embedded configuration file and the start-up configuration * ``nbtb config `` to show the start-up configuration for a specific tool Processing Multiple Notebooks ----------------------------- When more than one notebook is given, or in verbose mode, the output related to each notebook ``nb.ipynb`` is preceded by a header of the form .. code-block:: none ::::::::::::: nb.ipynb ::::::::::::: Some data is collected over all notebooks and aggregated (also see the option ``output-json`` to :ref:`output-json` and the **Examples** below). Common Tool Options ------------------- Many options, also for the tools themselves, have both a positive and a negated form. This is useful, because option values can also be read from configuration files. The following options apply to multiple tools (and many of them apply to all tools): .. code-block:: none -h, --help show this help message and exit -v, --verbose, -V, --no-verbose verbose mode produces extra output (default: False) -q, --quiet, -Q, --no-quiet quiet mode produces less output (default: False) --assert, --no-assert assert mode: when processing fails, abort with exit code 1 (default: True) --validate, --no-validate validate notebook before processing (default: False) --run, --no-run run notebook before processing (default: False) --inplace, --no-inplace replace original notebooks with processing result (default: False) --write-files, --no-write-files do write result files (default: True) -n short for --no-write-files: do processing but do not write result files (dry run) --config FILE.json read configuration from FILE.json (in JSON) --output-json FILE.json write statistical output to FILE.json -d, --debug debug mode produces diagnostic output (default: False) Help ^^^^ Each command provides its own help option: * ``nbtb -h`` or ``nbtb command --help`` The default option values shown, are after loading of the start-up configuration (see below). Verbose Mode ^^^^^^^^^^^^ Each command supprts verbose mode: * ``nbtb -v`` or ``nbtb --verbose`` In verbose mode, the tool's option values are shown, and the total number of processed notebooks is shown at the end, e.g.: .. code-block:: none Notebooks processed: 2 Quiet Mode ^^^^^^^^^^ Each command supprts quiet mode: * ``nbtb -q`` or ``nbtb --quiet`` In quiet mode, output is reduced to a minimum. Assert Mode ^^^^^^^^^^^ In assert mode (option ``-a`` or ``--assert``), whenever an error occurs during processing, all further processing will be interrupted and the tool aborts. Validate Mode ^^^^^^^^^^^^^ In validate mode (option ``--validate``), each notebook is validated before processing. Validation options can be set under the specific tool in a configuration file, but not on the command line. Run Mode ^^^^^^^^ In run mode (option ``--run``), each notebook is run before processing (and after validation, if that applies). Run options can be set under the specific tool in a configuration file, but not on the command line. Inplace ^^^^^^^ For some tools (currently: ``clean`` and ``run``), the option ``--inplace`` can be used to *replace* the input notebook with the resulting notebook. Write Files and Dry Run ^^^^^^^^^^^^^^^^^^^^^^^ In dry-run mode (option ``--no-write-files``, or ``-n`` for short), all processing and reporting is done, but resulting notebooks are not written to their destination file. This allows one te experiment with options. In particular, this is useful in combination with verbose mode. .. _configuration-file: Read Configuration File ^^^^^^^^^^^^^^^^^^^^^^^ Initial option settings can be read from a configuration file (option ``--config FILE.json``). At start up, each tool gathers an initial configuration as follows. #. The configuration file embedded in the package is loaded. #. If present, the configuration file ``/etc/nbtoolbelt.json`` is loaded on top of this. #. Finally, if present, the configuration file ``~/.nbtoolbelt.json`` is loaded on top of that. Here, ``~`` stands for the user's home directory. The invokation .. code-block:: bash $ nbtb config [tool] will show which configuration files are loaded, and what the resulting configuration is. Each tool can read options from a configuration file: * ``nbtb ... --config file.json ...`` The options from the configuration file override the preceding command-line options, and are, in turn, overridden by any subsequent command-line options. See :doc:`configuration` for details about configuration files. .. _output-json: Write JSON Output ^^^^^^^^^^^^^^^^^ Most tools gather some statistics as they process notebooks. These statistics are printed as notebooks are processed. For use with other tooling, such tool output can also be written to a file in `JSON `__ format (option ``--output-json FILE.json``). This JSON output file contains one *array*, where * item 0 concerns data aggregated over all notebooks that were processed; * item *i* (*i* from 1) concerns data collected for the *i*-th notebook argument; * each item is an object with further data appearing in various members, documented per tool; * one common member is present with name ``"file_name"``, and the file's name as value. Debug Mode ^^^^^^^^^^ In debug mode, diagnostic output is produced, and in particular, all option settings are shown. Available Tools --------------- The following tools are available: * ``nbtb validate ...`` :doc:`nbvalidate` * ``nbtb head ...`` :doc:`nbhead` * ``nbtb dump ...`` :doc:`nbdump` * ``nbtb stats ...`` :doc:`nbstats` * ``nbtb view ...`` :doc:`nbview` * ``nbtb cat ...`` :doc:`nbcat` * ``nbtb clean ...`` :doc:`nbclean` * ``nbtb run ...`` :doc:`nbrun` * ``nbtb split ...`` :doc:`nbsplit` * ``nbtb punch ...`` :doc:`nbpunch` Examples -------- The following invocation shows * the options loaded at start up from the various configuration files, * the location where these files were found, and * what the resulting configuration is: .. code-block:: bash $ nbtb config run Loading configuration for nbrun from: /Users/.../nbtoolbelt/data/nbtoolbelt.json Skipping configuration in: /etc/nbtoolbelt.json (not present) Loading configuration for nbrun from: /Users/.../.nbtoolbelt.json Configuration accumulated for nbrun: allow_errors (bool): True append_cell (bool): False appended_cell (str): '# Automatically added code cell: lists all global names defined by this notebook.\n%whos' assert (bool): True clean_after (bool): True clean_after_metadata (list): ['collapsed', 'scrolled'] clean_before (bool): True clean_before_metadata (list): ['ExecuteTime', 'execution'] debug (bool): False inplace (bool): False interrupt_on_timeout (bool): True ipc (str): '' kernel_name (str): '' notebooks (list): [] output_json (NoneType): None quiet (bool): False record_timing (bool): True run (bool): False run_path (str): '' run_result_name (str): '-run' streams_head (int): -1 streams_truncate_message (str): '*** Output truncated ***' timeout (int): -1 validate (bool): False verbose (bool): False write_files (bool): True The following invocation prints statistics for two notebooks, and writes JSON output to file ``nbtb-stats-output.json``. .. code-block:: bash $ nbtb stats short.ipynb test.ipynb --output-json nbtb-stats-output.json :::::::::::::: short.ipynb :::::::::::::: Notebook metadata: 4.2 format version python3 kernel python 3.6.1 language Cell types: 1 code 1 markdown 2 total cell count :::::::::::::: test.ipynb :::::::::::::: Notebook metadata: 4.2 format version python3 kernel python 3.6.1 language Cell types: 12 code 7 markdown 19 total cell count Totals ====== Cell types: 13 code 8 markdown 21 total cell count The file ``nbtb-stats-output.json`` then contains: .. code-block:: json [ { "cell_types": { "code": { "25%": 3.75, "50%": 6.5, "75%": 9.25, "count": 2.0, "max": 12.0, "mean": 6.5, "min": 1.0, "std": 7.7781745930520225, "total": 13.0 }, "markdown": { "25%": 2.5, "50%": 4.0, "75%": 5.5, "count": 2.0, "max": 7.0, "mean": 4.0, "min": 1.0, "std": 4.242640687119285, "total": 8.0 }, "total cell count": { "25%": 6.25, "50%": 10.5, "75%": 14.75, "count": 2.0, "max": 19.0, "mean": 10.5, "min": 2.0, "std": 12.020815280171307, "total": 21.0 } }, "file_count": 2, "file_name": { "count": 2, "freq": 1, "top": "short.ipynb", "unique": 2 }, "notebook_metadata": { "format version": { "count": 2, "freq": 2, "top": "4.2", "unique": 1 }, "kernel": { "count": 2, "freq": 2, "top": "python3", "unique": 1 }, "language": { "count": 2, "freq": 2, "top": "python 3.6.1", "unique": 1 } } }, { "cell_types": { "code": 1, "markdown": 1, "total cell count": 2 }, "file_name": "short.ipynb", "notebook_metadata": { "format version": "4.2", "kernel": "python3", "language": "python 3.6.1" } }, { "cell_types": { "code": 12, "markdown": 7, "total cell count": 19 }, "file_name": "test.ipynb", "notebook_metadata": { "format version": "4.2", "kernel": "python3", "language": "python 3.6.1" } } ]