Release notes for PyTables 3.2 series

Author:PyTables Developers
Contact:pytables-dev@googlegroups.com

Changes from 3.1.1 to 3.2.0

Improvements

  • The nrowsinbuf is better computed now for EArray/CArray having a small chunkshape in the main dimension. Fixes #285.

  • PyTables should be installable very friendly via pip, including NumPy being installed automatically in the unlikely case it is not yet installed in the system. Thanks to Andrea Bedini.

  • setup.py has been largely simplified and now it requires setuptools. Although we think this is a good step, please keep us informed this is breaking some installation in a very bad manner.

  • setup.py now is able to used pkg-config, if available, to locate required libraries (hdf5, bzip2, etc.). The use of pkg-config can be controlled via setup.py command line flags or via environment variables. Please refer to the installation guide (in the User Manual) for details. Closes gh-442.

  • It is now possible to create a new node whose parent is a softlink to another group (see gh-422). Thanks to Alistair Muldal.

  • link.SoftLink objects no longer need to be explicitly dereferenced. Methods and attributes of the linked object are now automatically accessed when the user acts on a soft-link (see gh-399). Thanks to Alistair Muldal.

  • Now ptrepack recognizes hardlinks and replicates them in the output (repacked) file. This saves disk space and makes repacked files more conformal to the original one. Closes gh-380.

  • New pttree script for printing HDF5 file contents as a pretty ASCII tree (closes gh-400). Thanks to Alistair Muldal.

  • The internal Blosc library has been downgraded to version 1.4.4. This is in order to still allow using multiple threads inside Blosc, even on multithreaded applications (see gh-411, gh-412, gh-437 and gh-448).

  • The print_versions() function now also reports the version of compression libraries used by Blosc.

  • Now the setup.py tries to use the ‘-march=native’ C flag by default. In falls back on ‘-msse2’ if ‘-march=native’ is not supported by the compiler. Closes gh-379.

  • Fixed a spurious unicode comparison warning (closes gh-372 and gh-373).

  • Improved handling of empty string attributes. In previous versions of PyTables empty string were stored as scalar HDF5 attributes having size 1 and value ‘0’ (an empty null terminated string). Now empty string are stored as HDF5 attributes having zero size

  • Added a new cookbook recipe and a couple of examples for simple threading with PyTables.

  • The redundant utilsextension.get_indices() function has been eliminated (replaced by slice.indices()). Closes gh-195.

  • Allow negative indices in point selection (closes gh-360)

  • Index wasn’t being used if it claimed there were no results. Closes gh-351 (see also gh-353)

  • Atoms and Col types are no longer generated dynamically so now it is easier for IDEs and static analysis tool to handle them (closes gh-345)

  • The keysort functions in idx-opt.c have been cythonised using fused types. The perfomance is mostly unchanged, but the code is much more simpler now. Thanks to Andrea Bedini.

  • Small unit tests re-factoring:

    • print_versions() and tests.common.print_heavy() functions

      moved to the tests.common module

    • always use print_versions() when test modules are called as scripts

    • use the unittest2 package in Python 2.6.x

    • removed internal machinery used to replicate unittest2 features

    • always use tests.common.PyTablesTestCase as base class for all test cases

    • code of the old tasts.common.cleanup() function has been moved to tests.common.PyTablesTestCase.tearDown() method

    • new implementation of tests.common.PyTablesTestCase.assertWarns() compatible with the one provided by the standard unittest module in Python >= 3.2

    • use tests.common.PyTablesTestCase.assertWarns() as context manager when appropriate

    • use the unittest.skipIf() decorator when appropriate

    • new :class:tests.comon.TestFileMixin: class

Bugs fixed

  • Fixed compatibility problems with numpy 1.9 and 1.10-dev (closes gh-362 and gh-366)
  • Fixed compatibility with Cython >= 0.20 (closes gh-386 and gh-387)
  • Fixed support for unicode node names in LRU cache (only Python 2 was affected). Closes gh-367 and gh-369.
  • Fixed support for unicode node titles (only Python 2 was affected). Closes gh-370 and gh-374.
  • Fixed a bug that caused the silent truncation of unicode attributes containing the ‘0’ character. Closes gh-371.
  • Fixed descr_from_dtype() to work as expected with complex types. Closes gh-381.
  • Fixed the tests.test_basics.ThreadingTestCase test case. Closes gh-359.
  • Fix incomplete results when performing the same query twice and exhausting the second iterator before the first. The first one writes incomplete results to seqcache (gh-353)
  • Fix false results potentially going to seqcache if tableextension.Row.update() is used during iteration (see gh-353)
  • Fix Column.create_csindex() when there’s NaNs
  • Fixed handling of unicode file names on windows (closes gh-389)
  • No longer not modify sys.argv at import time (closes gh-405)
  • Fixed a performance issue on NFS (closes gh-402)
  • Fixed a nasty problem affecting results of indexed queries. Closes gh-319 and probably gh-419 too.
  • Fixed another problem affecting results of indexed queries too. Closes gh-441.
  • Replaced “len(xrange(start, stop, step))” -> “len(xrange(0, stop - start, step))” to fix issues with large row counts with Python 2.x. Fixes #447.

Other changes

  • Cython is not a hard dependency anymore (although developers will need it so as to generated the C extension code).

  • The number of threads used by default for numexpr and Blosc operation that was set to the number of available cores have been reduced to 2. This is a much more reasonable setting for not creating too much overhead.

    Enjoy data!

    – The PyTables Developers