In copper, data processing is implemented through a
A pipeline is a series of processing routines for transforming raw input data
(e.g. electrophysiological data such as EMG) into useful output, such as the
velocity of a cursor on the screen. These routines can usually be broken down
into blocks which have common functionality.
The typical picture for an electrophysiological signal processing pipeline looks something like:
Input ↓ ┌──────────────────────┐ │ Windowing │ └──────────────────────┘ ↓ ┌──────────────────────┐ │ Conditioning │ └──────────────────────┘ ↓ ┌──────────────────────┐ │ Feature Extraction │ └──────────────────────┘ ↓ ┌──────────────────────┐ │ Intent Recognition │ └──────────────────────┘ ↓ ┌──────────────────────┐ │ Output Mapping │ └──────────────────────┘ ↓ Output
Each block in this example is really a type of processing block, and the actual processing involved in each can vary. copper implements some of the common cases, but creating custom blocks and connecting them together in a pipeline structure is simple. Also, the picture above shows a simple series structure, where each block takes input only from the block before it. More complex structures are sometimes convenient or necessary, and some complexity is supported.
Windowing involves specifying a time window over which the rest of the pipeline will operate. That is, a windower keeps track of the current input data and optionally some data from the past, concatentating the two and passing it along. This is useful for calculating statistics over a sufficient sample size while updating the pipeline output at a rapid rate, achieved by overlapping windows. In an offline processing context (i.e. processing static recordings), windowing also specifies how much data to read in on each iteration through the recording.
Windowing is handled by a
Raw data conditioning (or pre-processing) usually involves things like filtering and normalization. Usually the output of a conditioning block does not fundamentally change the representation of the input.
Features are statistics computed on a window of input data. Generally, they should represent the information contained in the raw input in a compact way. For example, you might take 100 samples of data from six channels of EMG and calculate the root-mean-square value of each channel during that 100-sample window of time. This results in an array of length 6 which represents the amplitude of each channel in the high-dimensional raw data. A feature extractor is just a collection of features to compute from the input.
Features in copper are classes that take all of their parameters in
__init__ and perform their operation on the input in a
Intent recognition is the prediction or estimation of what the user intends to do based on the signals generated. An example would be a large signal sensed at the group of extensor muscles in the forearm for communicating “wrist extension.” Sometimes this mapping can be specified a priori, but most of the time we rely on machine learning techniques to infer this mapping from training data.
core module is a small infrastructure for processing data in
a pipeline style. You create or use the built-in
objects, then connect them up with an efficient (but still readable) syntax
The syntax for expressing pipeline structure is based on lists and tuples. Lists hold elements that are connected in series:
[a, b]: ─a─b─
The input is whatever
a takes, and the output is whatever
Tuples hold elements that are connected in parallel:
(a, b): ┌─a─┐ ─┤ ┝━ └─b─┘
The input goes to both
b, and the output is whatever
b output in a list. If we connect another element in series with a parallel
block, it must be a block that handles multiple inputs:
[(a, b), c]: ┌─a─┐ ─┤ ┝━c─ └─b─┘
The bottom line is: pipeline blocks accept input types and they specify the output types. You are responsible for ensuring that pipeline blocks can be connected as specified.
Sometimes, you might want to pass the output of a block to some block structure
and somewhere downstream. To handle this case, there is
PassthroughPipeline that you can use as a block within another
passthrough pipeline p ← (b, c): ┌─────┐ ├─b─┐ │ ─┤ ┝━┷━ └─c─┘ [a, p, d]: ┌─────┐ ├─b─┐ │ ─a─p━d─ → ─a─┤ ┝━┷━d─ └─c─┘
The pass-through pipeline places its own output(s) after its input, so the input is accesible on the other side. There are cases where this type of structure is possible with a list/tuple expression, but sometimes the pass-through pipeline as a block is needed. The above example is one of those cases.
Implementing Pipeline Blocks¶
Pipeline blocks are simple to implement. It is only expected that you implement
process() method which takes one argument (
data) and returns
something. For multi-input blocks, you’ll probably want to just expand the
inputs right off the bat (e.g.
in_a, in_b = data). Usually, the output is
some processed form of the input data:
import copper class FooBlock(copper.PipelineBlock): def process(self, data): return data + 1 class BarBlock(copper.PipelineBlock): def process(self, data): return 2 * data
With some blocks implemented, the list/tuple syntax described above is used for specifying how they are connected:
a = FooBlock() b = BarBlock() p = copper.Pipeline([a, b])
Now, you just give the pipeline input and get its output:
input = 3 result = p.process(input)
In this case, the result would be
2 * (input + 1) == 8.
Sometimes, it’s useful to be able to hook into some block in the pipeline to retrieve its data in the middle of a run through the pipeline. For instance, let’s say you have a simple pipeline:
[a, b]: ─a─b─
You run some data through the pipeline to get the result from block
you also want to run some function with the output of
hooks keword argument which takes a list of functions to execute
after the block’s
process method finishes. To use hooks, make sure your
custom block calls the parent
__init__ method. For
import copper class FooBlock(copper.PipelineBlock): def __init__(self, hooks=None): super(FooBlock, self).__init__(hooks=hooks) def process(self, data): return data + 1 class BarBlock(copper.PipelineBlock): def process(self, data): return 2 * data def foo_hook(data): print("FooBlock output is %d".format(data)) a = FooBlock(hooks=[foo_hook]) b = BarBlock() p = copper.Pipeline([a, b]) result = p.process(3)
Now, the call to
process on the pipeline will input 3 to block
a will add 1 then print
FooBlock output is 4, and then 4 will be passed
b, which will return 8.