|
| 1 | +# IPython/Jupyter Extensions |
| 2 | + |
| 3 | +The `pyrasterframes.rf_ipython` module injects a number of visualization extensions into the IPython environment, enhancing visualization of `DataFrame`s and `Tile`s. |
| 4 | + |
| 5 | +By default, the last expression's result in a IPython cell is passed to the `IPython.display.display` function. This function in turn looks for a [`DisplayFormatter`](https://ipython.readthedocs.io/en/stable/api/generated/IPython.core.formatters.html#IPython.core.formatters.DisplayFormatter) associated with the type, which in turn converts the instance to a display-appropriate representation, based on MIME type. For example, each `DisplayFormatter` may `plain/text` version for the IPython shell, and a `text/html` version for a Jupyter Notebook. |
| 6 | + |
| 7 | +This will be our setup for the following examples: |
| 8 | + |
| 9 | +```python setup |
| 10 | +from pyrasterframes import * |
| 11 | +from pyrasterframes.rasterfunctions import * |
| 12 | +from pyrasterframes.utils import create_rf_spark_session |
| 13 | +import pyrasterframes.rf_ipython |
| 14 | +from IPython.display import display |
| 15 | +import os.path |
| 16 | +spark = create_rf_spark_session() |
| 17 | +def scene(band): |
| 18 | + b = str(band).zfill(2) # converts int 2 to '02' |
| 19 | + return 'https://modis-pds.s3.amazonaws.com/MCD43A4.006/11/08/2019059/' \ |
| 20 | + 'MCD43A4.A2019059.h11v08.006.2019072203257_B{}.TIF'.format(b) |
| 21 | +rf = spark.read.raster(scene(2), tile_dimensions=(256, 256)) |
| 22 | +``` |
| 23 | + |
| 24 | +## Tile Samples |
| 25 | + |
| 26 | +We have some convenience methods to quickly visualize tiles (see discussion of the RasterFrame @ref:[schema](raster-read.md#single-raster) for orientation to the concept) when inspecting a subset of the data in a Notebook. |
| 27 | + |
| 28 | +In an IPython or Jupyter interpreter, a `Tile` object will be displayed as an image with limited metadata. |
| 29 | + |
| 30 | +```python, sample_tile |
| 31 | +sample_tile = rf.select(rf_tile('proj_raster').alias('tile')).first()['tile'] |
| 32 | +sample_tile # or `display(sample_tile)` |
| 33 | +``` |
| 34 | + |
| 35 | +## DataFrame Samples |
| 36 | + |
| 37 | +Within an IPython or Jupyter interpreter, a Spark and Pandas DataFrames containing a column of _tiles_ will be rendered as the samples discussed above. Simply import the `rf_ipython` submodule to enable enhanced HTML rendering of these DataFrame types. |
| 38 | + |
| 39 | +```python display_samples |
| 40 | +rf # or `display(rf)`, or `rf.display()` |
| 41 | +``` |
| 42 | + |
| 43 | +### Changing Number of Rows |
| 44 | + |
| 45 | +By default the RasterFrame sample display renders 5 rows. Because the `IPython.display.display` function doesn't pass parameters to the underlying rendering functions, we have to provide a different means of passing parameters to the rendering code. Pandas approach to this is to use global settings via `set_option`/`get_option`. We take a more functional approach and have the user invoke an explicit `display` method: |
| 46 | + |
| 47 | +```python custom_display, evaluate=False |
| 48 | +rf.display(num_rows=1, truncate=True) |
| 49 | +``` |
| 50 | + |
| 51 | +```python custom_display_mime, echo=False |
| 52 | +rf.display(num_rows=1, truncate=True, mimetype='text/markdown') |
| 53 | +``` |
| 54 | + |
| 55 | +### Pandas |
| 56 | + |
| 57 | +There is similar rendering support injected into the Pandas by the `rf_ipython` module, for Pandas Dataframes having Tiles in them: |
| 58 | + |
| 59 | +```python pandas_dataframe |
| 60 | +# Limit copy of data from Spark to a few tiles. |
| 61 | +pandas_df = rf.select(rf_tile('proj_raster'), rf_extent('proj_raster')).limit(4).toPandas() |
| 62 | +pandas_df # or `display(pandas_df)` |
| 63 | +``` |
| 64 | + |
| 65 | +## Sample Colorization |
| 66 | + |
| 67 | +RasterFrames uses the "Viridis" color ramp as the default color profile for tile column. There are other options for reasoning about how color should be applied in the results. |
| 68 | + |
| 69 | +### Color Composite |
| 70 | + |
| 71 | +As shown in @ref:[Writing Raster Data section](raster-write.md) section, composites can be constructed for visualization: |
| 72 | + |
| 73 | +```python, png_color_composite |
| 74 | +from IPython.display import Image # For telling IPython how to interpret the PNG byte array |
| 75 | +# Select red, green, and blue, respectively |
| 76 | +three_band_rf = spark.read.raster(source=[[scene(1), scene(4), scene(3)]]) |
| 77 | +composite_rf = three_band_rf.withColumn('png', |
| 78 | + rf_render_png('proj_raster_0', 'proj_raster_1', 'proj_raster_2')) |
| 79 | +png_bytes = composite_rf.select('png').first()['png'] |
| 80 | +Image(png_bytes) |
| 81 | +``` |
| 82 | + |
| 83 | +```python, png_render, echo=False |
| 84 | +from IPython.display import display_markdown |
| 85 | +display_markdown(pyrasterframes.rf_ipython.binary_to_html(png_bytes), raw=True) |
| 86 | +``` |
| 87 | + |
| 88 | +### Custom Color Ramp |
| 89 | + |
| 90 | +You can also apply a different color ramp to a single-channel Tile using the @ref[`rf_render_color_ramp_png`](reference.md#rf-render-color-ramp-png) function. See the function documentation for information about the available color maps. |
| 91 | + |
| 92 | +```python, color_map |
| 93 | +rf.select(rf_render_color_ramp_png('proj_raster', 'Magma')) |
| 94 | +``` |
0 commit comments