simpletable.legacy package

Submodules

simpletable.legacy.simpletable module

This file implements a Table class

that is designed to be the basis of any format

Requirements

• FIT format:

  • astropy:

    provides a replacement to pyfits pyfits can still be used instead but astropy is now the default

• HDF5 format:

  • pytables

RuntimeError will be raised when writing to a format associated with missing package.

class AstroHelpers

Bases: object

Helpers related to astronomy data

static conesearch(ra0, dec0, ra, dec, r, outtype=0)

Perform a cone search on a table

Parameters:

• ra0 (ndarray[ndim=1, dtype=float]) – column name to use as RA source in degrees

• dec0 (ndarray[ndim=1, dtype=float]) – column name to use as DEC source in degrees

• ra (float) – ra to look for (in degree)

• dec (float) – ra to look for (in degree)

• r (float) – distance in degrees

• outtype (int) –

  type of outputs

  0 – minimal, indices of matching coordinates 1 – indices and distances of matching coordinates 2 – full, boolean filter and distances

Returns:

t –

if outtype is 0:

only return indices from ra0, dec0

elif outtype is 1:

return indices from ra0, dec0 and distances

elif outtype is 2:

return conditional vector and distance to all ra0, dec0

Return type:

tuple

static deg2dms(val, delim=':')

Convert degrees into hex coordinates

Parameters:

• deg (float) – angle in degrees

• delimiter (str) – character delimiting the fields

Returns:

str – string to convert

Return type:

string or sequence

static deg2hms(val, delim=':')

Convert degrees into hex coordinates

Parameters:

• deg (float) – angle in degrees

• delimiter (str) – character delimiting the fields

Returns:

str – string to convert

Return type:

string or sequence

static dms2deg(_str, delim=':')

Convert hex coordinates into degrees

Parameters:

• str (string or sequence) – string to convert

• delimiter (str) – character delimiting the fields

Returns:

deg – angle in degrees

Return type:

float

static euler(ai_in, bi_in, select, b1950=False, dtype='f8')

Transform between Galactic, celestial, and ecliptic coordinates. Celestial coordinates (RA, Dec) should be given in equinox J2000 unless the b1950 is True.

select	From	To	select	From	To
1	RA-Dec (2000)	Galactic	4	Ecliptic	RA-Dec
2	Galactic	RA-DEC	5	Ecliptic	Galactic
3	RA-Dec	Ecliptic	6	Galactic	Ecliptic

Parameters:

• long_in (float, or sequence) – Input Longitude in DEGREES, scalar or vector.

• lat_in (float, or sequence) – Latitude in DEGREES

• select (int) – Integer from 1 to 6 specifying type of coordinate transformation.

• b1950 (bool) – set equinox set to 1950

Returns:

• long_out (float, seq) – Output Longitude in DEGREES

• lat_out (float, seq) – Output Latitude in DEGREES

Note

Written W. Landsman, February 1987 Adapted from Fortran by Daryl Yentis NRL Converted to IDL V5.0 W. Landsman September 1997 Made J2000 default, added /FK4 keyword W. Landsman December 1998 Add option SELECT as a keyword W. Landsman March 2003 Converted from IDL to Python: Erin Sheldon, NYU, 2008-07-02

static hms2deg(_str, delim=':')

Convert hex coordinates into degrees

Parameters:

• str (string or sequence) – string to convert

• delimiter (str) – character delimiting the fields

Returns:

deg – angle in degrees

Return type:

float

static sphdist(ra1, dec1, ra2, dec2)

measures the spherical distance between 2 points

Parameters:

• ra1 (float or sequence) – first right ascensions in degrees

• dec1 (float or sequence) – first declination in degrees

• ra2 (float or sequence) – second right ascensions in degrees

• dec2 (float or sequence) – first declination in degrees

Returns:

Outputs – returns a distance in degrees

Return type:

float or sequence

class AstroTable

Bases: SimpleTable

Derived from the Table, this class add implementations of common astro tools especially conesearch

__init__(*args, **kwargs)

coneSearch(ra, dec, r, outtype=0)

Perform a cone search on a table

Parameters:

• ra0 (ndarray[ndim=1, dtype=float]) – column name to use as RA source in degrees

• dec0 (ndarray[ndim=1, dtype=float]) – column name to use as DEC source in degrees

• ra (float) – ra to look for (in degree)

• dec (float) – ra to look for (in degree)

• r (float) – distance in degrees

• outtype (int) –

  type of outputs

  0 – minimal, indices of matching coordinates 1 – indices and distances of matching coordinates 2 – full, boolean filter and distances

Returns:

t –

if outtype is 0:

only return indices from ra0, dec0

elif outtype is 1:

return indices from ra0, dec0 and distances

elif outtype is 2:

return conditional vector and distance to all ra0, dec0

Return type:

tuple

get_DEC(degree=True)

Returns RA, converted from hexa/sexa into degrees

get_RA(degree=True)

Returns RA, converted from hexa/sexa into degrees

info()

prints information on the table

selectWhere(fields, condition=None, condvars=None, cone=None, zone=None, **kwargs)

Read table data fulfilling the given condition. Only the rows fulfilling the condition are included in the result.

conesearch is also possible through the keyword cone formatted as (ra, dec, r)

zonesearch is also possible through the keyword zone formatted as (ramin, ramax, decmin, decmax)

Combination of multiple selections is also available.

set_DEC(val)

Set the column that defines DEC coordinates

set_RA(val)

Set the column that defines RA coordinates

where(condition=None, condvars=None, cone=None, zone=None, **kwargs)

Read table data fulfilling the given condition. Only the rows fulfilling the condition are included in the result.

Parameters:

• condition (str) – expression to evaluate on the table includes mathematical operations and attribute names

• condvars (dictionary, optional) – A dictionary that replaces the local operands in current frame.

Returns:

• out (ndarray/ tuple of ndarrays)

• result equivalent to np.where()

zoneSearch(ramin, ramax, decmin, decmax, outtype=0)

Perform a zone search on a table, i.e., a rectangular selection

Parameters:

• ramin (float) – minimal value of RA

• ramax (float) – maximal value of RA

• decmin (float) – minimal value of DEC

• decmax (float) – maximal value of DEC

• outtype (int) –

  type of outputs

  0 or 1 – minimal, indices of matching coordinates 2 – full, boolean filter and distances

Returns:

r – indices or conditional sequence of matching values

Return type:

sequence

class SimpleTable

Bases: object

Table class that is designed to be the basis of any format wrapping around numpy recarrays

fname

if str, the file to read from. This may be limited to the format currently handled automatically. If the format is not correctly handled, you can try by providing an object.__

if object with a structure like dict, ndarray, or recarray-like

the data will be encapsulated into a Table

Type:

str or object

caseless

if set, column names will be caseless during operations

Type:

bool

aliases

set of column aliases (can be defined later set_alias())

Type:

dict

units

set of column units (can be defined later set_unit())

Type:

dict

desc

set of column description or comments (can be defined later set_comment())

Type:

dict

header

key, value pair corresponding to the attributes of the table

Type:

dict

property Plotter

Plotter instance related to this dataset. Requires plotter add-on to work

__init__(fname, *args, **kwargs)

addCol(name, data, dtype=None, unit=None, description=None)

Add one or multiple columns to the table

Parameters:

• name (str or sequence(str)) – The name(s) of the column(s) to add

• data (ndarray, or sequence of ndarray) – The column data, or sequence of columns

• dtype (dtype) – numpy dtype for the data to add

• unit (str) – The unit of the values in the column

• description (str) – A description of the content of the column

addLine(iterable)

Append one row in this table.

see also: stack()

Parameters:

iterable (iterable) – line to add

add_column(name, data, dtype=None, unit=None, description=None)

Add one or multiple columns to the table

Parameters:

• name (str or sequence(str)) – The name(s) of the column(s) to add

• data (ndarray, or sequence of ndarray) – The column data, or sequence of columns

• dtype (dtype) – numpy dtype for the data to add

• unit (str) – The unit of the values in the column

• description (str) – A description of the content of the column

append_row(iterable)

Append one row in this table.

see also: stack()

Parameters:

iterable (iterable) – line to add

property colnames

Sequence of column names

compress(condition, axis=None, out=None)

Return selected slices of an array along given axis.

When working along a given axis, a slice along that axis is returned in output for each index where condition evaluates to True. When working on a 1-D array, compress is equivalent to extract.

Parameters:

• condition (1-D array of bools) – Array that selects which entries to return. If len(condition) is less than the size of a along the given axis, then output is truncated to the length of the condition array.

• axis (int, optional) – Axis along which to take slices. If None (default), work on the flattened array.

• out (ndarray, optional) – Output array. Its type is preserved and it must be of the right shape to hold the output.

Returns:

compressed_array – A copy of a without the slices along axis for which condition is false.

Return type:

ndarray

delCol(names)

Remove several columns from the table

Parameters:

names (sequence) – A list containing the names of the columns to remove

property dtype

dtype of the data

property empty_row

Return an empty row array respecting the table format

evalexpr(expr, exprvars=None, dtype=<class 'float'>)

evaluate expression based on the data and external variables

all np function can be used (log, exp, pi…)

Parameters:

• expr (str) – expression to evaluate on the table includes mathematical operations and attribute names

• exprvars (dictionary, optional) – A dictionary that replaces the local operands in current frame.

• dtype (dtype definition) – dtype of the output array

Returns:

out – array of the result

Return type:

NumPy array

find_duplicate(index_only=False, values_only=False)

Find duplication in the table entries, return a list of duplicated elements Only works at this time is 2 lines are the same entry not if 2 lines have the same values

get(v, full_match=False)

returns a table from columns given as v

this function is equivalent to __getitem__() but preserve the Table format and associated properties (units, description, header)

Parameters:

• v (str) – pattern to filter the keys with

• full_match (bool) – if set, use re.fullmatch() instead of re.match()

groupby(*key)

Create an iterator which returns (key, sub-table) grouped by each value of key(value)

Parameters:

key (str) – expression or pattern to filter the keys with

Returns:

• key (str or sequence) – group key

• tab (SimpleTable instance) – sub-table of the group header, aliases and column metadata are preserved (linked to the master table).

info()

prints information on the table

items()

Iterator on the (key, value) pairs

iterkeys()

Iterator over the columns of the table

itervalues()

Iterator over the lines of the table

join(other, on=None, left_on=None, right_on=None, lsuffix='', rsuffix='', how='left', inplace=False)

Return a dataset joined with other datasets, matched

by columns/expression on/left_on/right_on

If neither on/left_on/right_on is given, the join is done by simply adding the columns (i.e. on the implicit row index).

Parameters:

• other (Table) – Table to join with

• on (string) – default key for the left table (self)

• left_on (string) – key for the left table (self), overrides on

• right_on (string) – default key for the right table (other), overrides on

• lsuffix (string) – suffix to add to the left column names in case of a name collision

• rsuffix (string) – similar for the right

• how (string) – how to join, ‘left’ keeps all rows on the left, and adds columns (with possible missing values) ‘right’ is similar with self and other swapped.

• inplace (boolean) – do not copy the left table

Returns:

tab – joined table

Return type:

Table

keys(regexp=None, full_match=False, skip_aliases=False)

Return the data column names or a subset of it

Parameters:

• regexp (str) – pattern to filter the keys with

• full_match (bool) – if set, use re.fullmatch() instead of re.match()

• skip_aliases (bool) – include aliases if unset

• string (Try to apply the pattern at the start of the)

• returning

• object (a match)

• found. (or None if no match was)

Returns:

seq – sequence of keys

Return type:

sequence

match(r2, key)

Returns the indices at which the tables match matching uses 2 columns that are compared in values

Parameters:

• r2 (Table) – second table to use

• key (str) – fields used for comparison.

Returns:

indexes – tuple of both indices list where the two columns match.

Return type:

tuple

property name

name of the table given by the Header[‘NAME’] attribute

property nbytes

number of bytes of the object

property ncols

number of columns

property nrows

number of lines

pop_columns(names)

Pop several columns from the table

Parameters:

names (sequence) – A list containing the names of the columns to remove

Returns:

values – list of columns

Return type:

tuple

pprint(idx=None, fields=None, ret=False, all=False, full_match=False, headerChar='-', delim=' | ', endline='\n', **kwargs)

Pretty print the table content

you can select the table parts to display using idx to select the rows and fields to only display some columns (ret is only for insternal use)

Parameters:

• idx (sequence, slide) – sub selection to print

• fields (str, sequence) – if str can be a regular expression, and/or list of fields separated by spaces or commas

• ret (bool) – if set return the string representation instead of printing the result

• all (bool) – if set, force to show all rows

• headerChar (char) – Character to be used for the row separator line

• delim (char) – The column delimiter.

pprint_entry(num, keys=None)

print one line with key and values properly to be readable

Parameters:

• num (int, slice) – indice selection

• keys (sequence or str) – if str, can be a regular expression if sequence, the sequence of keys to print

remove_column(names)

Remove several columns from the table

Parameters:

names (sequence) – A list containing the names of the columns to remove

remove_columns(names)

Remove several columns from the table

Parameters:

names (sequence) – A list containing the names of the columns to remove

rename_columns(**mapping)

Rename the fields from a flexible-datatype ndarray or recarray.

Parameters:

mapping (dictionary) – Dictionary mapping old field names to their new version.

resolve_alias(colname)

Return the name of an aliased column.

Given an alias, return the column name it aliases. This function is a no-op if the alias is a column name itself.

Aliases are defined by using .define_alias()

reverse_alias(colname)

Return aliases of a given column.

Given a colname, return a sequence of aliases associated to this column Aliases are defined by using .define_alias()

select(fields, indices=None, **kwargs)

Select only a few fields in the table

Parameters:

• fields (str or sequence) – fields to keep in the resulting table

• indices (sequence or slice) – extract only on these indices

Returns:

tab – resulting table

Return type:

SimpleTable instance

selectWhere(fields, condition, condvars=None, **kwargs)

Read table data fulfilling the given condition.

Only the rows fulfilling the condition are included in the result.

Parameters:

• fields (str or sequence) – fields to keep in the resulting table

• condition (str) – expression to evaluate on the table includes mathematical operations and attribute names

• condvars (dictionary, optional) – A dictionary that replaces the local operands in current frame.

Returns:

tab – resulting table

Return type:

SimpleTable instance

setComment(colname, comment)

Set the comment of a column referenced by its name

Parameters:

• colname (str) – column name or registered alias

• comment (str) – column description

setUnit(colname, unit)

Set the unit of a column referenced by its name

Parameters:

• colname (str) – column name or registered alias

• unit (str) – unit description

set_alias(alias, colname)

Define an alias to a column

Parameters:

• alias (str) – The new alias of the column

• colname (str) – The column being aliased

set_comment(colname, comment)

Set the comment of a column referenced by its name

Parameters:

• colname (str) – column name or registered alias

• comment (str) – column description

set_unit(colname, unit)

Set the unit of a column referenced by its name

Parameters:

• colname (str) – column name or registered alias

• unit (str) – unit description

property shape

shape of the data

sort(keys, copy=False)

Sort the table inplace according to one or more keys. This operates on the existing table (and does not return a new table).

Parameters:

• keys (str or seq(str)) – The key(s) to order by

• copy (bool) – if set returns a sorted copy instead of working inplace

stack(r, *args, **kwargs)

Superposes arrays fields by fields inplace

t.stack(t1, t2, t3, default=None, inplace=True)

Parameters:

r (Table)

stats(fn=None, fields=None, fill=None)

Make statistics on columns of a table

Parameters:

• fn (callable or sequence of callables) – functions to apply to each column default: (np.mean, np.std, np.nanmin, np.nanmax)

• fields (str or sequence) – any key or key expression to subselect columns default is all columns

• fill (value) – value when not applicable default np.nan

Returns:

tab – collection of statistics, one column per function in fn and 1 ligne per column in the table

Return type:

Table instance

take(indices, axis=None, out=None, mode='raise')

Take elements from an array along an axis.

This function does the same thing as “fancy” indexing (indexing arrays using arrays); however, it can be easier to use if you need elements along a given axis.

Parameters:

• indices (array_like) – The indices of the values to extract. Also allow scalars for indices.

• axis (int, optional) – The axis over which to select values. By default, the flattened input array is used.

• out (ndarray, optional) – If provided, the result will be placed in this array. It should be of the appropriate shape and dtype.

• mode ({'raise', 'wrap', 'clip'}, optional) –

  Specifies how out-of-bounds indices will behave.

  • ’raise’ – raise an error (default)

  • ’wrap’ – wrap around

  • ’clip’ – clip to the range

  ’clip’ mode means that all indices that are too large are replaced by the index that addresses the last element along that axis. Note that this disables indexing with negative numbers.

Returns:

subarray – The returned array has the same type as a.

Return type:

ndarray

to_astropy_table(**kwargs)

A class to represent tables of heterogeneous data.

astropy.table.Table provides a class for heterogeneous tabular data, making use of a numpy structured array internally to store the data values. A key enhancement provided by the Table class is the ability to easily modify the structure of the table by adding or removing columns, or adding new rows of data. In addition table and column metadata are fully supported.

Parameters:

• masked (bool, optional) – Specify whether the table is masked.

• names (list, optional) – Specify column names

• dtype (list, optional) – Specify column data types

• meta (dict, optional) – Metadata associated with the table.

• copy (bool, optional) – Copy the input data (default=True).

• rows (numpy ndarray, list of lists, optional) – Row-oriented data for table instead of data argument

• copy_indices (bool, optional) – Copy any indices in the input data (default=True)

• **kwargs (dict, optional) – Additional keyword args when converting table-like object

Returns:

df – dataframe

Return type:

astropy.table.Table

to_dask(**kwargs)

Construct a Dask DataFrame

This splits an in-memory Pandas dataframe into several parts and constructs a dask.dataframe from those parts on which Dask.dataframe can operate in parallel.

Note that, despite parallelism, Dask.dataframe may not always be faster than Pandas. We recommend that you stay with Pandas for as long as possible before switching to Dask.dataframe.

Parameters:

• keys (sequence, optional) – ordered subset of columns to export

• npartitions (int, optional) – The number of partitions of the index to create. Note that depending on the size and index of the dataframe, the output may have fewer partitions than requested.

• chunksize (int, optional) – The size of the partitions of the index.

• sort (bool) – Sort input first to obtain cleanly divided partitions or don’t sort and don’t get cleanly divided partitions

• name (string, optional) – An optional keyname for the dataframe. Defaults to hashing the input

Returns:

A dask DataFrame/Series partitioned along the index

Return type:

dask.DataFrame or dask.Series

to_dict(keys=None, contiguous=False)

Construct a dictionary from this dataframe with contiguous arrays

Parameters:

• keys (sequence, optional) – ordered subset of columns to export

• contiguous (boolean) – make sure each value is a contiguous numpy array object (C-aligned)

Returns:

data – converted data

Return type:

dict

to_pandas(**kwargs)

Construct a pandas dataframe

Parameters:

• data (ndarray) – (structured dtype), list of tuples, dict, or DataFrame

• keys (sequence, optional) – ordered subset of columns to export

• index (string, list of fields, array-like) – Field of array to use as the index, alternately a specific set of input labels to use

• exclude (sequence, default None) – Columns or fields to exclude

• columns (sequence, default None) – Column names to use. If the passed data do not have names associated with them, this argument provides names for the columns. Otherwise this argument indicates the order of the columns in the result (any names not found in the data will become all-NA columns)

• coerce_float (boolean, default False) – Attempt to convert values to non-string, non-numeric objects (like decimal.Decimal) to floating point, useful for SQL result sets

Returns:

df

Return type:

DataFrame

to_records(**kwargs)

Construct a numpy record array from this dataframe

to_vaex(**kwargs)

Create an in memory Vaex dataset

Parameters:

• name (str) – unique for the dataset

• keys (sequence, optional) – ordered subset of columns to export

Returns:

df – vaex dataset

Return type:

vaex.DataSetArrays

to_xarray(**kwargs)

Construct an xarray dataset

Each column will be converted into an independent variable in the Dataset. If the dataframe’s index is a MultiIndex, it will be expanded into a tensor product of one-dimensional indices (filling in missing values with NaN). This method will produce a Dataset very similar to that on which the ‘to_dataframe’ method was called, except with possibly redundant dimensions (since all dataset variables will have the same dimensionality).

where(condition, condvars=None, *args, **kwargs)

Read table data fulfilling the given condition. Only the rows fulfilling the condition are included in the result.

Parameters:

• condition (str) – expression to evaluate on the table includes mathematical operations and attribute names

• condvars (dictionary, optional) – A dictionary that replaces the local operands in current frame.

Returns:

• out (ndarray/ tuple of ndarrays)

• result equivalent to np.where()

write(fname, **kwargs)

write table into file

Parameters:

• fname (str) – filename to export the table into

• note:: (..) – additional keywords are forwarded to the corresponding libraries pyfits.writeto() or pyfits.append() np.savetxt()

class stats

Bases: object

static has_nan(v)

static max(v)

static mean(v)

static min(v)

static p16(v)

static p50(v)

static p84(v)

static std(v)

static var(v)

Module contents