{
"cells": [
{
"cell_type": "markdown",
"id": "16cc3741",
"metadata": {},
"source": [
"# Logging Custom Images\n",
"\n",
"**Welcome**\n",
"\n",
"To the third part of EncoderMap's customization notebooks.\n",
"\n",
"Run this notebook on Google Colab:\n",
"\n",
"[](https://colab.research.google.com/github/AG-Peter/encodermap/blob/main/tutorials/notebooks_customization/03_custom_images.ipynb)\n",
"\n",
"Find the documentation of EncoderMap:\n",
"\n",
"https://ag-peter.github.io/encodermap\n",
"\n",
"### For Google colab only:\n",
"\n",
"If you're on Google colab, please uncomment these lines and install EncoderMap."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "daa31ab5",
"metadata": {
"execution": {
"iopub.execute_input": "2024-12-29T12:55:14.510353Z",
"iopub.status.busy": "2024-12-29T12:55:14.510233Z",
"iopub.status.idle": "2024-12-29T12:55:14.513041Z",
"shell.execute_reply": "2024-12-29T12:55:14.512569Z"
}
},
"outputs": [],
"source": [
"# !wget https://gist.githubusercontent.com/kevinsawade/deda578a3c6f26640ae905a3557e4ed1/raw/b7403a37710cb881839186da96d4d117e50abf36/install_encodermap_google_colab.sh\n",
"# !sudo bash install_encodermap_google_colab.sh"
]
},
{
"cell_type": "markdown",
"id": "032d11e1",
"metadata": {},
"source": [
"**Goals:**\n",
"\n",
"In this tuorial you will learn how to add custom images to the \"Images\" section in TensorBoard. This can be done in two ways:\n",
"\n",
"- [Providing `EncoderMap.add_images_to_tensorboard()` with custom function.](#custom-fn)\n",
"- [Writing a custom Callback, that inherits from `encodermap.callbacks.EncoderMapBaseCallback`.](#custom-callback)\n",
"\n",
"As usual, we will start to import some packages. Along the usual packages we import the built-in package `io`."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "1e7ec075",
"metadata": {
"execution": {
"iopub.execute_input": "2024-12-29T12:55:14.515676Z",
"iopub.status.busy": "2024-12-29T12:55:14.515554Z",
"iopub.status.idle": "2024-12-29T12:55:18.841681Z",
"shell.execute_reply": "2024-12-29T12:55:18.840915Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/kevin/git/encoder_map_private/encodermap/__init__.py:194: GPUsAreDisabledWarning: EncoderMap disables the GPU per default because most tensorflow code runs with a higher compatibility when the GPU is disabled. If you want to enable GPUs manually, set the environment variable 'ENCODERMAP_ENABLE_GPU' to 'True' before importing EncoderMap. To do this in python you can run:\n",
"\n",
"import os; os.environ['ENCODERMAP_ENABLE_GPU'] = 'True'\n",
"\n",
"before importing encodermap.\n",
" _warnings.warn(\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "f52b6766b3ec47d1b4e3017fda6bf3a3",
"version_major": 2,
"version_minor": 0
},
"text/plain": []
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import numpy as np\n",
"import encodermap as em\n",
"import tensorflow as tf\n",
"import plotly.express as px\n",
"import plotly.graph_objects as go\n",
"from plotly.subplots import make_subplots\n",
"import pandas as pd\n",
"\n",
"import io\n",
"\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"id": "5ad5afb8",
"metadata": {},
"source": [
"We will use `io` to write a png-file to a buffer (not to disk) and provide that puffer to Tensorboard for visualization. But first, let us think about what to plot."
]
},
{
"cell_type": "markdown",
"id": "41613662",
"metadata": {},
"source": [
"\n",
"\n",
"## Logging via a custom function\n",
"\n",
"What shall we use as an example in this section?\n",
"Let's take the images, that EncoderMap automatically logs during training. These images are generated from a subset of the training data. This subset is passed through the encoder part of the network. The histogram is created from the point in the `_gen_hist_matplotlib()` function in the `encoderamp.misc.summaries.py` module."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "03fd886a-f8ea-40fe-9ca4-11aee0599c80",
"metadata": {
"execution": {
"iopub.execute_input": "2024-12-29T12:55:18.843870Z",
"iopub.status.busy": "2024-12-29T12:55:18.843569Z",
"iopub.status.idle": "2024-12-29T12:55:18.847100Z",
"shell.execute_reply": "2024-12-29T12:55:18.846712Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"def _gen_hist_matplotlib(\n",
" data: np.ndarray,\n",
" hist_kws: dict[str, Any],\n",
") -> tf.Tensor:\n",
" \"\"\"Creates matplotlib histogram and returns tensorflow Tensor that represents an image.\n",
"\n",
" Args:\n",
" data (Union[np.ndarray, tf.Tensor]): The xy data to be used. data.ndim should be 2.\n",
" 1st dimension the datapoints, 2nd dimension x, y.\n",
" hist_kws (dict): Additional keywords to be passed to matplotlib.pyplot.hist2d().\n",
"\n",
" Returns:\n",
" tf.Tensor: A tensorflow tensor that can be written to Tensorboard with tf.summary.image().\n",
"\n",
" \"\"\"\n",
" plt.close(\"all\")\n",
" matplotlib.use(\"Agg\") # overwrites current backend of notebook\n",
" plt.figure()\n",
" plt.hist2d(*data.T, **hist_kws)\n",
" buf = io.BytesIO()\n",
" plt.savefig(buf, format=\"png\")\n",
" buf.seek(0)\n",
" image = tf.image.decode_png(buf.getvalue(), 4)\n",
" image = tf.expand_dims(image, 0)\n",
" return image\n",
"\n"
]
}
],
"source": [
"from encodermap.misc.summaries import _gen_hist_matplotlib\n",
"import inspect\n",
"print(inspect.getsource(_gen_hist_matplotlib))"
]
},
{
"cell_type": "markdown",
"id": "bcaceeca-5762-4f08-8182-5a98860d0b10",
"metadata": {},
"source": [
"We can see, that the function that creates the histogram is rather simple. It takes a NumPy array (`data: np.ndarray`) and keyword arguments (`hist_kws: dict[str, Any]`) for matplotlib's `plt.hist2d()`. But what if we want to use the (x, y) data to plot the a free energy-representation of the 2D latent space. Let's develop such a function. We will use SKLearn's `make_blobs()` function to create the test data."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "1eed95e8-e1c1-4fab-9bb7-67359599acc4",
"metadata": {
"execution": {
"iopub.execute_input": "2024-12-29T12:55:18.849044Z",
"iopub.status.busy": "2024-12-29T12:55:18.848892Z",
"iopub.status.idle": "2024-12-29T12:55:19.187582Z",
"shell.execute_reply": "2024-12-29T12:55:19.186938Z"
}
},
"outputs": [
{
"data": {
"text/html": [
" \n",
" "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"plotlyServerURL": "https://plot.ly"
},
"data": [
{
"hovertemplate": "x=%{x}
y=%{y}