PyDIVIDE’s Documentation¶
Introduction¶
What is PyDIVIDE?¶
PyDIVIDE a toolkit for manipulating and visualizing data from the MAVEN mission. The toolkit started in IDL, but the core functionality has been ported over into python. The tool has primarily been focused on the Key Parameter dataset in the past, but due to the limited nature of that dataset will soon focus more on the Level 2 data.
It can plot using either Qt via pytqtgraph, or by creating HTML files via Bokeh.
What does it do?¶
There are a few major things the tool does:
- Downloads data files from the MAVEN Science Data Center
- Reads in data from Key Parameter data files
- Plots data in a variety of ways (Time, Altitude, Lat vs Lon, etc)
- Reads in data files from models of the Martian Atmosphere
- Provides time series data analysis/manipulation routines
Pyqtgraph Sample¶

Getting Started¶
System Requirements¶
The MAVEN PyDIVIDE toolkit currently requires Anaconda 5.0 or above. Anaconda will install Python, as well as numerous software libraries for scientific computing. This toolkit is only compatible with Python 3.5 or above.
To install the PyDIVIDE toolkit, type the following command into the local terminal/Anaconda Prompt terminal.
pip install pydivide
This will automatically install most of the required dependencies. The following is necessary for the Bokeh HTML plots:
conda install -c bokeh nodejs
And finally, though this is not required, for 3D visualization you will need to get PyOpenGL
pip install pyopengl
The PyDIVIDE toolkit can also be downloaded from the MAVEN Science Data Center GitHub page, https://github.com/MAVENSDC. This will require manual installation of all dependencies of PyDIVIDE. It is recommended that PyDIVIDE be installed via the pip command above.
Updating the Toolkit¶
The latest version of PyDIVIDE can be installed by typing the following command into the terminal
pip install pydivide --upgrade
Running PyDIVIDE¶
An IDE is the recommended way to run PyDIVIDE procedures; however, they can also be run from the terminal. To start an interactive session of Python, enter the following commands into the terminal
ipython
import pydivide
Data Storage¶
PyDIVIDE requires all data files to be stored in an automatically-created directory structure. This has a similar format to the SDC and SSL directory structures. The root directory for data storage can be chosen by the user. When first running a download_files or read procedure, the user will be prompted to select the root_data_dir. After the directory is selected, it is saved in mvn_toolkit_prefs.txt, and can later be changed manually as desired. After the first selection of the directory, the user will not be prompted by download_files or read again. download_files will place files into the chosen directory structure, and read will pull data files from that directory structure.
Note
If you have pyspedas installed (installed by default when installed via pip), then pydivide will use the pyspedas data directory. The default is C://Datapy, but can be changed with pyspedas.set_prefs(‘data_dir’, ‘/insert/path/for/your/data’)
While you do not necessarily need to know this to use the toolkit, the data directories are structured as such
- <root_data_dir>/maven/data/sci
- /kp/insitu/YYYY/MM/ /kp/iuvs/YYYY/MM/ /sta/l2/YYYY/MM/ /sep/l2/YYYY/MM/ /swi/l2/YYYY/MM/ /swe/l2/YYYY/MM/ /lpw/l2/YYYY/MM/ /mag/l2/YYYY/MM/ /iuv/l2/YYYY/MM/ /ngi/l2/YYYY/MM/ /euv/l2/YYYY/MM/ /acc/l2/YYYY/MM/
Reading in Data¶
This page will describe how to obtain and read in the data. For reading/loading Level 2 files, PySPEDAS is currently required.
Download Data¶
-
pydivide.
download_files
(filenames=None, instruments=None, list_files=False, level='l2', insitu=True, iuvs=False, new_files=False, start_date='2014-01-01', end_date='2020-01-01', update_prefs=False, only_update_prefs=False, exclude_orbit_file=False, local_dir=None, unittest=False, crustal_download=True)[source]¶ Download data files from the MAVEN SDC web server. Compatible with KP files or instrument-specific data downloads. insitu, iuvs, or at least one instrument must be specified.
Parameters: - filenames – str/list of str Specific filename strings to search/download.
- instruments – str/list of str - swe, swi, ngi, euv, lpw, iuv, rse, sta, sep, acc Instruments from which you want to download data.
- list_files – bool (True/False) If true, lists the files instead of downloading them.
- level – str Data level to download.
- insitu – bool (True/False) If true, specifies only insitu files.
- iuvs – bool (True/False) If true,
- new_files – bool (True/False) Checks downloaded files and only downloads those that haven’t already been downloaded.
- start_date – str String that is the start date for downloading data (YYYY-MM-DD)
- end_date – str String that is the end date for downloading data (YYYY-MM-DD)
- update_prefs – bool (True/False) If true, updates where you want to store data locally
- only_update_prefs – bool (True/False) If true, only updates where to store dat alocally, doesn’t download files.
- exclude_orbit_file – bool (True/False) If true, won’t download the latest orbit tables.
- local_dir – str If indicated, specifies where to download files for a specific implementation of this function.
- unittest – bool If True, will not actually download files. If False (default) files will be downloaded.
- crustal_download – bool If True (default), when insitu files are downloaded, any crustal files will also be downloaded. If False, crustal files will not be downloaded when insitu files are downloaded.
Returns: None
Examples
>>> # Download all available insitu data between 2015-01-01 and 2015-01-31, inclusive: >>> pydivide.download_files(start_date='2015-01-01', end_date='2015-01-31', insitu=True)
>>> # List all available CDF insitu KP files on the server: >>> pydivide.download_files(insitu=True, list_files=True)
>>> # Download all new IUVS files from 6 April 2015 not found in the local directory. >>> pydivide.download_files(iuvs=True, new_files=True, end_date='2015-04-06')
>>> # List all available Level 2 data files for SWIA. >>> pydivide.download_files(instruments='swi', list_files=True, level='l2')
>>> # List all available Level 2 data files for SWIA for the month of January 2015. >>> pydivide.download_files(start_date='2015-01-01', end_date='2015-01-31', instruments='swi', list_files=True, level='l2')
>>> # Download all new Level 2 data files for NGIMS, STATIC, and EUV. >>> pydivide.download_files(instruments=['ngi','sta','euv'], new_files=True)
Read KP Data¶
-
pydivide.
read
(filename=None, input_time=None, instruments=None, insitu_only=False, specified_files_only=False)[source]¶ Read in a given filename in situ file into a dictionary object Optional keywords maybe used to downselect instruments returned and the time windows.
Parameters: - filename – str/list of str Name of the in situ KP file(s) to read in.
- input_time – list of str/int Set a time bounds/filter on the data, must be length 2 with the first value being the start time, and the second value being the end time.
- instruments – Optional keyword listing the instruments to include in the returned dictionary/structure.
- insitu_only – Optional keyword that allows you to specify that you only want to download insitu files.
- specified_files_only – Optional keyword that allows you to specify you only want filenames given in ‘filename’ to be read in, not other files close in date/time as well.
Returns: A dictionary (data structure) containing up to all of the columns included in a MAVEN in-situ Key parameter data file.
Examples
>>> # Retrieve insitu and IUVS data for LPW and MAG on 2015-12-26. >>> insitu,iuvs = pydivide.read('2015-12-26', instruments=['lpw','mag'])
>>> # Retrieve only insitu data for all instruments on 2017-06-19. >>> insitu = pydivide.read('2017-06-19', insitu_only=True)
Read Model Results¶
-
pydivide.
read_model_results
(file)[source]¶ Reads results of specified simulation into a dictionary object containing sub-directories for metadata, dimension information, and model tracers. This function can read any of the models currently on the MAVEN SDC website with the .nc extension, which can be found here: https://lasp.colorado.edu/maven/sdc/public/pages/models.html The desired model must be downloaded prior to running this procedure
Parameters: file – str Simulation result file name to be read Returns: - Dictionary roughly structured as follows
- _ meta ()
- _ longsubsol _ ls _ etc
- _ dim
- _ lat/x _ lon/y _ alt/z
- _ variable1
- _ dim_order (x,y,z or z,y,x for example) _ data
- _ variable2
- _ dim_order _ data
… _ variableN
Examples
>>> # Read the University of Michigan group’s ionospheric model for mean solar activity (F10.7 = 130). >>> model = pydivide.read_model_results('<dir_path>/MGITM_LS270_F130_150519.nc')
Plotting Insitu KP Data¶
This page will describe how to plot the KP data
Altitiude Plots¶
-
pydivide.
altplot
(kp, parameter=None, time=None, errors=None, sameplot=True, list=False, title='Altitude Plot', ylog=False, qt=True)[source]¶ Plot the provided data plotted against spacecraft altitude. If time is not provided plot entire data set.
Parameters: - kp – dict insitu kp data structure/dictionary read from file(s)
- parameter – list of str/int The parameter(s) to be plotted. Can be provided as integers (by index) or strings (by name: inst.obs). If a single parameter is provided, it must be an int or str. If several are provided it must be a list. A list may contain a mixture of data types.
- time – list of str Two-element list of strings or integers indicating the range of Time to be plotted. At present, there are no checks on whether provided Times are within provided data
- sameplot – bool if True, put all curves on same axes if False, generate new axes for each plot
- list – bool Lists all Key Parameters instead of plotting
- title – str The Title to give the plot
- ylog – bool Displays the log of the y axis
- qt – bool If true, plots with qt. Else creates an HTML page with bokeh.
Returns: None
Examples
>>> # Plot LPW.ELECTRON_DENSITY against spacecraft altitude. >>> pydivide.altplot(insitu, parameter=['LPW.ELECTRON_DENSITY','MAG.MSO_Y'], qt=False, ylog=True)
Time Series Plots¶
-
pydivide.
plot
(kp, parameter=None, time=None, errors=None, sameplot=True, list=False, title='', qt=True, exec_qt=True, log=False)[source]¶ Plot time-series data from insitu data structure.
Parameters: - kp – dict insitu kp data structure/dictionary read from file(s)
- parameter – list of str/int The parameter(s) to be plotted. Can be provided as integers (by index) or strings (by name: inst.obs). If a single parameter is provided, it must be an int or str. If several are provided it must be a list. A list may contain a mixture of data types.
- time – list of str Two-element list of strings or integers indicating the range of Time to be plotted. At present, there are no checks on whether provided Times are within provided data
- sameplot – bool if True, put all curves on same axes if False, generate new axes for each plot
- list – bool Lists all Key Parameters instead of plotting
- title – str The Title to give the plot
- qt – bool If true, plots with qt. Else creates an HTML page with bokeh.
- exec_qt – bool If False, does not run the event loop for pyqtgraph.
- Returns :
- None
Examples
>>> # Plot SWIA H+ density. >>> pydivide.plot(insitu,parameter='swia.hplus_density') >>> # Plot SWIA H+ density and altitude in the same window. >>> pydivide.plot(insitu,parameter=['swia.hplus_density', 'spacecraft.altitude'],sameplot=True)
Standard Plots¶
-
pydivide.
standards
(kp, list_plots=False, all_plots=False, euv=False, mag_mso=False, mag_geo=False, mag_cone=False, mag_dir=False, ngims_neutral=False, ngims_ions=False, eph_angle=False, eph_geo=False, eph_mso=False, swea=False, sep_ion=False, sep_electron=False, wave=False, plasma_den=False, plasma_temp=False, swia_h_vel=False, static_h_vel=False, static_o2_vel=False, static_flux=False, static_energy=False, sun_bar=False, solar_wind=False, ionosphere=False, sc_pot=False, altitude=False, title='Standard Plots', qt=True)[source]¶ Generate all or a subset of 25 standardized plots, created from insitu KP data on the MAVEN SDC website
Parameters: - kp – dict insitu kp data structure/dictionary read from file(s)
- mag_mso – bool magnetic field, MSO coordinates
- mag_geo – bool magnetic field, geographic coordinates
- mag_cone – bool magnetic clock and cone angles, MSO coordinates
- mag_dir – bool magnetic field, radial/horizontal/northward/eastward components
- ngims_neutral – bool neutral atmospheric component densities
- ngims_ions – bool ionized atmospheric component densities
- eph_angle – bool spacecraft ephemeris information
- eph_geo – bool spacecraft position, geographic coordinates
- eph_mso – bool spacecraft position, MSO coordinates
- swea – bool electron parallel/anti-parallel fluxes
- sep_ion – bool ion energy flux
- sep_electron – bool electron energy flux
- wave – bool electric field wave power
- plasma_den – bool plasma density
- plasma_temp – bool plasma temperature
- swia_h_vel – bool H+ flow velocity, SWIA MSO coordinates
- static_h_vel – bool H+ flow velocity, STATIC MSO coordinates
- static_o2_vel – bool O2+ flow velocity, STATIC MSO coordinates
- static_flux – bool H+/He++ and pick-up ion omni-directional flux
- static_energy – bool H+/He++ and pick-up ion characteristic energy
- sun_bar – bool MAVEN sunlight indicator
- solar_wind – bool solar wind dynamic pressure
- ionosphere – bool electron spectrum shape parameter
- altitude – bool spacecraft altitude
- sc_pot – bool spacecraft potential
- list – bool Lists all Key Parameters instead of plotting
- title – str The Title to give the plot
- qt – bool If true, plots with qt. Else creates an HTML page with bokeh.
- exec_qt – bool If False, does not run the event loop for pyqtgraph.
- Returns :
- None
- Examples :
>>> # Solar Orbital coordinates (x, y, z, magnitude), standard spacecraft ephemeris >>> # information (sub-spacecraft lat/lon, subsolar lat/lon, local solar time, solar >>> # zenith angle, Mars season) >>> # omni-directional flux. >>> insitu,iuvs = pydivide.read(input_time=['2017-06-19','2017-06-20']) >>> pydivide.standards(insitu, mag_mso=True, eph_angle=True, title='Example Title')