Unum
- Ebuilds: 1, Testing: 4.2.1 Description:
If you make scientific or engineering calculations it is a best practice to always include units when you define variables if they are relevant. For example, if you define velocity to be 100, and time to be 0.5, Python cannot perform unit checking or unit balancing when you calculate the distance. It is a better practice to define the velocity as 100 km/h and the time as 30 min. It can also prevent errors later on. There are few ‘Famous Unit Conversion Errors’ that occurred because of unit calculation issue.
Homepage:
https://bitbucket.org/lukaszlaba/unum
https://unum.readthedocs.io/en/latest/
License: GPL-3
algopy
- Ebuilds: 1, Testing: 0.5.7 Description:
AlgoPy allows you to differentiate functions implemented as computer programs by using Algorithmic Differentiation (AD) techniques in the forward and reverse mode.
The forward mode propagates univariate Taylor polynomials of arbitrary order. Hence it is also possible to use AlgoPy to evaluate higher-order derivative tensors.
Speciality of AlgoPy is the possibility to differentiate functions that contain matrix functions as +,-,*,/, dot, solve, qr, eigh, cholesky.
Homepage:
https://github.com/b45ch1/algopy
License: BSD
aspectlib
- Ebuilds: 1, Testing: 2.0.0 Description:
An aspect-oriented programming, monkey-patch and decorators library. It is useful when changing behavior in existing code is desired. It includes tools for debugging and testing: simple mock/record and a complete capture/replay framework.
Homepage:
https://github.com/ionelmc/python-aspectlib
https://pypi.org/project/python-aspectlib/
License: BSD
correctionlib
- Ebuilds: 2, Testing: 2.4.0 Description:
The purpose of this library is to provide a well-structured JSON data format for a wide variety of ad-hoc correction factors encountered in a typical HEP analysis and a companion evaluation tool suitable for use in C++ and python programs. Here we restrict our definition of correction factors to a class of functions with scalar inputs that produce a scalar output.
Homepage:https://github.com/cms-nanoAOD/correctionlib/ License: BSD
dask-awkward
- Ebuilds: 2, Testing: 2023.12.0 Description:
Awkward Array is a library for nested, variable-sized data, including arbitrary-length lists, records, mixed types, and missing data, using NumPy-like idioms.
Homepage:https://github.com/dask-contrib/dask-awkward/ License: BSD
dm-tree
- Ebuilds: 1, Testing: 0.1.8 Description:
tree is a library for working with nested data structures. In a way, tree generalizes the builtin map function which only supports flat sequences, and allows to apply a function to each "leaf" preserving the overall structure.
Homepage:https://github.com/deepmind/tree License: MIT
histoprint
- Ebuilds: 1, Testing: 2.4.0 Description:
Histoprint uses a mix of terminal color codes and Unicode trickery (i.e. combining characters) to plot overlaying histograms. Some terminals are not able to display Unicode combining characters correctly. Histoprint can still be used in those terminals, but the character set needs to be constrained to the non-combining ones (see below).
Homepage:https://github.com/scikit-hep/histoprint License: MIT
hunter
- Ebuilds: 2, Testing: 3.6.1 Description:
Hunter is a flexible code tracing toolkit, not for measuring coverage, but for debugging, logging, inspection and other nefarious purposes. It has a simple Python API, a convenient terminal API and a CLI tool to attach to processes.
Design notes
Hunter doesn't do everything. As a design goal of this library some things are made intentionally austere and verbose (to avoid complexity, confusion and inconsistency). This has few consequences:
There are Operators but there's no negation operator. Instead you're expected to negate a Query object, eg: ~Q(module='re').
There are no specialized operators or filters - all filters behave exactly the same. For example:
No filter for packages. You're expected to filter by module with an operator.
No filter for arguments, return values or variables. You're expected to write your own filter function and deal with the problems of poking into objects.
Layering is minimal. There's are some helpers that do some argument processing and conversions to save you some typing but that's about it.
The library doesn't try to hide the mechanics of tracing in Python - it's 1:1 regarding what Python sends to a trace function if you'd be using sys.settrace.
Doesn't have any storage. You are expected to redirect output to a file.
You should look at it like it's a tool to help you understand and debug big applications, or a framework ridding you of the boring parts of settrace, not something that helps you learn Python.
FAQ
Why not Smiley?
There's some obvious overlap with smiley but there are few fundamental differences:
Complexity. Smiley is simply over-engineered:
It uses IPC and a SQL database.
It has a webserver. Lots of dependencies.
It uses threads. Side-effects and subtle bugs are introduced in your code.
It records everything. Tries to dump any variable. Often fails and stops working.
Why do you need all that just to debug some stuff in a terminal? Simply put, it's a nice idea but the design choices work against you when you're already neck-deep into debugging your own code. In my experience Smiley has been very buggy and unreliable. Your mileage may vary of course.
Tracing long running code. This will make Smiley record lots of data, making it unusable.
Now because Smiley records everything, you'd think it's better suited for short programs. But alas, if your program runs quickly then it's pointless to record the execution. You can just run it again.
It seems there's only one situation where it's reasonable to use Smiley: tracing io-bound apps remotely. Those apps don't execute lots of code, they just wait on network so Smiley's storage won't blow out of proportion and tracing overhead might be acceptable.
Use-cases. It seems to me Smiley's purpose is not really debugging code, but more of a "non interactive monitoring" tool.
In contrast, Hunter is very simple:
Few dependencies.
Low overhead (tracing/filtering code has an optional Cython extension).
No storage. This simplifies lots of things.
The only cost is that you might need to run the code multiple times to get the filtering/actions right. This means Hunter is not really suited for "post-mortem" debugging. If you can't reproduce the problem anymore then Hunter won't be of much help.
Why not pytrace?
Pytrace is another tracer tool. It seems quite similar to Smiley - it uses a sqlite database for the events, threads and IPC, thus it's reasonable to expect the same kind of problems.
Why not PySnooper or snoop?
snoop is a refined version of PySnooper. Both are more suited to tracing small programs or functions as the output is more verbose and less suited to the needs of tracing a big application where Hunter provides more flexible setup, filtering capabilities, speed and brevity.
Why not coverage?
For purposes of debugging coverage is a great tool but only as far as "debugging by looking at what code is (not) run". Checking branch coverage is good but it will only get you as far.
From the other perspective, you'd be wondering if you could use Hunter to measure coverage-like things. You could do it but for that purpose Hunter is very "rough": it has no builtin storage. You'd have to implement your own storage. You can do it but it wouldn't give you any advantage over making your own tracer if you don't need to "pre-filter" whatever you're recording.
In other words, filtering events is the main selling point of Hunter - it's fast (cython implementation) and the query API is flexible enough.
Homepage:
https://github.com/ionelmc/python-hunter
https://pypi.org/project/hunter/
License: BSD-2
klepto
- Ebuilds: 2, Testing: 0.2.4 Description:
klepto extends python's lru_cache to utilize different keymaps and alternate caching algorithms, such as lfu_cache and mru_cache. While caching is meant for fast access to saved results, klepto also has archiving capabilities, for longer-term storage. klepto uses a simple dictionary-sytle interface for all caches and archives, and all caches can be applied to any python function as a decorator. Keymaps are algorithms for converting a function's input signature to a unique dictionary, where the function's results are the dictionary value. Thus for y = f(x), y will be stored in cache[x] (e.g. {x:y}).
Homepage:https://github.com/uqfoundation/klepto License: BSD
llvmlite
- Ebuilds: 3, Testing: 0.41.1 Description:
llvmlite uses the LLVM library for JIT (just-in-time) compilation of
Python code into native machine instructions during runtime. Instead
of exposing large parts of the LLVM C++ API for direct calls into the
LLVM library, llvmlite follows a lightweight multi-layered approach.
Homepage:https://llvmlite.pydata.org/ License: BSD
manhole
- Ebuilds: 1, Testing: 1.8.0-r1 Description:
Manhole is in-process service that will accept unix domain socket connections and present the stacktraces for all threads and an interactive prompt. It can either work as a python daemon thread waiting for connections at all times or a signal handler (stopping your application and waiting for a connection).
Access to the socket is restricted to the application's effective user id or root.
This is just like Twisted's manhole. It's simpler (no dependencies), it only runs on Unix domain sockets (in contrast to Twisted's manhole which can run on telnet or ssh) and it integrates well with various types of applications.
Homepage:
https://github.com/ionelmc/python-manhole
https://pypi.org/project/manhole/
License: BSD-2
multiprocess
- Ebuilds: 2, Testing: 0.70.15 Description:
multiprocess is a fork of multiprocessing. multiprocess extends multiprocessing to provide enhanced serialization, using dill. multiprocess leverages multiprocessing to support the spawning of processes using the API of the python standard library's threading module. multiprocessing has been distributed as part of the standard library since python 2.6.
Homepage:https://github.com/uqfoundation/multiprocess License: BSD
mystic
- Ebuilds: 2, Testing: 0.4.1 Description:
The mystic framework provides a collection of optimization algorithms and tools that allows the user to more robustly (and easily) solve hard optimization problems. All optimization algorithms included in mystic provide workflow at the fitting layer, not just access to the algorithms as function calls. mystic gives the user fine-grained power to both monitor and steer optimizations as the fit processes are running. Optimizers can advance one iteration with Step, or run to completion with Solve. Users can customize optimizer stop conditions, where both compound and user-provided conditions may be used. Optimizers can save state, can be reconfigured dynamically, and can be restarted from a saved solver or from a results file. All solvers can also leverage parallel computing, either within each iteration or as an ensemble of solvers.
Homepage:https://github.com/uqfoundation/mystic License: BSD
papermill
- Ebuilds: 2, Testing: 2.6.0 Description:
papermill is a tool for parameterizing, executing, and analyzing Jupyter Notebooks.
Homepage:https://github.com/nteract/papermill/ License: MIT
pathos
- Ebuilds: 2, Testing: 0.3.1 Description:
pathos is a framework for heterogeneous computing. It provides a consistent high-level interface for configuring and launching parallel computations across heterogeneous resources. pathos provides configurable launchers for parallel and distributed computing, where each launcher contains the syntactic logic to configure and launch jobs in an execution environment. Examples of launchers that plug into pathos are: a queue-less MPI-based launcher (in pyina), a ssh-based launcher (in pathos), and a multi-process launcher (in multiprocess).
Homepage:https://github.com/uqfoundation/pathos License: BSD
pox
- Ebuilds: 2, Testing: 0.3.3 Description:
pox provides a collection of utilities for navigating and manipulating filesystems. This module is designed to facilitate some of the low level operating system interactions that are useful when exploring a filesystem on a remote host, where queries such as “what is the root of the filesystem?”, “what is the user’s name?”, and “what login shell is preferred?” become essential in allowing a remote user to function as if they were logged in locally. While pox is in the same vein of both the os and shutil builtin modules, the majority of its functionality is unique and compliments these two modules.
Homepage:https://github.com/uqfoundation/pox License: BSD
ppft
- Ebuilds: 2, Testing: 1.7.6.7 Description:
ppft is a friendly fork of Parallel Python (pp). ppft extends Parallel Python to provide packaging and distribution with pip and setuptools, support for python 3, and enhanced serialization using dill.source. ppft uses Parallel Python to provide mechanisms for the parallel execution of python code on SMP (systems with multiple processors or cores) and clusters (computers connected via network).
Homepage:https://github.com/uqfoundation/ppft License: BSD
pqdm
- Ebuilds: 1, Testing: 0.2.0 Description:
PQDM is a TQDM and concurrent futures wrapper to allow enjoyable paralellization of iterating through an Iterable with a progress bar.
Homepage:https://github.com/niedakh/pqdm License: MIT
pyina
- Ebuilds: 1, Testing: 0.2.8 Description:
The pyina package provides several basic tools to make MPI-based parallel computing more accessable to the end user. The goal of pyina is to allow the user to extend their own code to MPI-based parallel computing with minimal refactoring.
Homepage:
https://github.com/uqfoundation/pyina
https://pypi.org/project/pyina/
License: BSD
resample
- Ebuilds: 1, Testing: 1.6.0 Description:
Resampling-based inference in Python based on data resampling and permutation.
Homepage:https://github.com/scikit-hep/resample License: BSD
scrapbook
- Ebuilds: 1, Testing: 0.5.0 Description:
The scrapbook library records a notebook’s data values and generated visual content as "scraps". Recorded scraps can be read at a future time.
Homepage:https://github.com/nteract/scrapbook/ License: MIT