adding titles to TOC and landing pages for everything other than Euclid by jkrick · Pull Request #220 · Caltech-IPAC/irsa-tutorials

jkrick · 2026-01-22T17:51:05Z

closes #217

This PR should

generate content for landing pages for each category of notebooks which has more than one notebook
update toc.yml to including landing pages
update toc.yml to use titles when referencing notebooks - (https://mystmd.org/guide/table-of-contents#document-titles)

along the way, I have

renamed "Special Topics" to "Techniques & Tools"
made the notebook descriptions very short in preparation for cards instead of lists on the landing pages.

I did not

rename the notebooks in line with their short titles, however tempting this is
re-organize the repo directory structure to bring it in line with the toc, however tempting this is

jkrick · 2026-01-22T17:55:47Z

@bsipocz I am adding short titles to the irsa tutorials per #217 . Should I also change the names of the actual .md tutorial files to match those short titles. Would make sense to me, but I don't see the whole picture. I put this draft up as an example. So in this case, I would recommend re-naming spherex_intro.md to data_overview.md to match the short title I chose, and also for the other tutorials. I didn't choose "spherex intro" as the short title (which would have simplified this) because I think having a table of contents with "spherex" as the main header and then under that something that repeats the term "spherex" is not a good idea.

jkrick · 2026-01-22T18:04:22Z

@bsipocz @troyraen Also, the directory structure of irsa-tutorials doesn't match the table of contents structure. Should I change that to be in line with each other? ie., in the table of contents on https://caltech-ipac.github.io/irsa-tutorials/ we have a "WISE" header, but there is no wise directory under irsa-tutorials/tutorials/ This is because we changed the table of contents to be mission specific ~1 year ago. How deep into this can of worms do you want me to dig? What other monsters will I unleash if I keep digging?

bsipocz · 2026-01-22T18:06:56Z

tutorials/spherex/spherex.md

+SPHEREx IRSA data releases include weekly [Quick Release spectral image products](https://caltech-ipac.github.io/spherex-archive-documentation/spherex-data-products/) (multi-extension FITS files containing calibrated near-infrared surface brightness, variance, flags, modeled backgrounds, PSFs, and wavelength WCS) along with ancillary calibration and metadata files such as gain matrices, dark current maps, solid angle pixel maps, and detailed spectral WCS products for each detector.
+
+- [Data Overview](https://caltech-ipac.github.io/irsa-tutorials/spherex-intro/) - Introduces the SPHEREx mission data available at IRSA, explaining the structure of the data products and how to identify which files and formats are appropriate for different science use cases.
+
+- [Spectral Image Cutouts](https://caltech-ipac.github.io/irsa-tutorials/spherex-cutouts/) - Demonstrates how to generate and work with spatial and spectral cutouts from SPHEREx data, enabling users to extract subsets of the all-sky survey tailored to specific targets or regions.
+
+- [PSF Models](https://caltech-ipac.github.io/irsa-tutorials/spherex-psf/) - Explains how SPHEREx point spread function (PSF) information is organized and accessed, and shows how PSF models can be used to interpret, model, or forward-simulate SPHEREx observations.


Don't use full render URLs, but internal references. The labels you deleted below should be the ones

bsipocz · 2026-01-22T18:07:01Z

tutorials/spherex/spherex_cutouts.md

  name: python3
 ---

-(spherex-cutouts)=


bsipocz · 2026-01-22T18:07:07Z

tutorials/spherex/spherex_intro.md

  name: python3
 ---

-(spherex-intro)=


bsipocz · 2026-01-22T18:15:23Z

@bsipocz I am adding short titles to the irsa tutorials per #217 . Should I also change the names of the actual .md tutorial files to match those short titles. Would make sense to me, but I don't see the whole picture. I put this draft up as an example. So in this case, I would recommend re-naming spherex_intro.md to data_overview.md to match the short title I chose, and also for the other tutorials. I didn't choose "spherex intro" as the short title (which would have simplified this) because I think having a table of contents with "spherex" as the main header and then under that something that repeats the term "spherex" is not a good idea.

OK, so this will be a multitude of a few things.

no need to rename the files, the filename and short title doesn't have to be connected, the short title is just the text that will show up in the TOC on the left hand side
- Jupyter Lab/hub/notebook apparently doesn't like the myst metadata. I don't yet have a bug report for this, but it seems that we either be very careful with committing any changes to the metadata after editing notebooks in JH (Use query_ssa() for Euclid tutorials 3 & 5 #216 (comment)) or find another solution

2*) with the very latest mystmd version released yesterday it is now possible to assign short titles in the toc file. So, I think we should do this as it is both more correct (that has effect on the TOC/rendering only not on the notebook itself) and also works around the bug mentioned above.

If you want to rename files for any reasons (e.g. you would find it easier to navigate and find them in JH) then we can do that. I have no opinions of what they should be called (but having unique names would help); but the caveat is that we need to make sure they are renamed everywhere -- in the manifest file, and in any cross references where we may still use file names rather than internal labels.

jkrick · 2026-01-22T18:28:43Z

Jupyter Lab/hub/notebook apparently doesn't like the myst metadata. I don't yet have a bug report for this, but it seems that we either be very careful with committing any changes to the metadata after editing notebooks in JH (#216 (comment)) or find another solution

ugh, really? I looked at these, and the only change I saw is the one I wrote with short_title. Which I did after following #206 Unfortunately this is a learning process of which things are problems and which aren't.

2*) with the very latest mystmd version released yesterday it is now possible to assign short titles in the toc file. So, I think we should do this as it is both more correct (that has effect on the TOC/rendering only not on the notebook itself) and also works around the bug mentioned above.

can you tell me how to do this? or an example?

bsipocz · 2026-01-22T18:54:21Z

ugh, really? I looked at these, and the only change I saw is the one I wrote with short_title. Which I did after following #206 Unfortunately this is a learning process of which things are problems and which aren't.

This is all good, the rendering and everything should be fine, but if you open these in jupyter and run them I believe the short_title entries will get deleted. That happened in your euclid PR at least.

bsipocz · 2026-01-22T18:58:32Z

2*) with the very latest mystmd version released yesterday it is now possible to assign short titles in the toc file. So, I think we should do this as it is both more correct (that has effect on the TOC/rendering only not on the notebook itself) and also works around the bug mentioned above.

can you tell me how to do this? or an example?

What you have here will work for now, but going fwd I think we will need to do this instead:
https://mystmd.org/guide/table-of-contents#document-titles

The myst version containing this change has already been released (you need 1.8.0 for this -- released yesterday), but we use jupyter-book (which is a thin wrapper) and it is still in the middle of the release process. So it won't yet work for us here in CI. So keep doing what you do here, you shouldn't be blocked I just mention that we have a solution/workaround that we can switch to in the near future.

jkrick · 2026-01-22T19:02:59Z

I just mention that we have a solution/workaround that we can switch to in the near future.

Any idea what "near future" means? I don't want to touch this twice, so Im happy to leave it alone if there will be a better solution in the next few weeks.

bsipocz · 2026-01-22T19:06:11Z

Any idea what "near future" means?

yesterday :) No, I mean the release was planned to be done early this week, and most of it was done yesterday. So I would say, this should all work by next week?
(as I don't see a reason why we shouldn't require the latest release; beside this TOC issue I also expect the new release to solve most of our memory problems on circleCI)

troyraen

Thanks for working on this; we really need it! This content will be very visible, so please don't merge before soliciting feedback from the full team and iterating until consensus. (I see this is marked as draft, so maybe that's already your plan.) For now, I read through and commented on a couple things that stuck out, but not a detailed review.

the directory structure of irsa-tutorials doesn't match the table of contents structure. Should I change that to be in line with each other?

I think we probably should. But this will be a big change, so I prefer that we touch base about it with the full team first. Then if we decide to move forward, make a PR dedicated to that change rather than mixing it in with adding these landing pages.

tutorials/techniques.md

toc.yml

bsipocz · 2026-01-26T18:02:49Z

@jkrick - you can now add "title" to the toc.yml file, which is equivalent to the short_title in the metadata. (See #224 for euclid)

…ration for cards

jkrick

@troyraen I was indeed not finished with this PR, but had put it up as a draft to get some thoughts on directions it needed to go before declaring it ready for review.

I think it is now in a good enough state to be ready for review, so please give me your feedback.

tutorials/techniques.md

bsipocz

I'm leaving a request changes status on this, so we don't accidentally forget to remove the temporarily commit.

jkrick · 2026-01-26T19:48:33Z

@bsipocz the "title" in the toc isn't showing up in the Rendering Preview. Those are still showing up with some longer text. What am I missing?

bsipocz

The suggestions should fix the issue

toc.yml

tutorials/wise.md

bsipocz · 2026-01-26T20:04:55Z

the "title" in the toc isn't showing up in the Rendering Preview. Those are still showing up with some longer text. What am I missing?

Added a commit that should fix it up. TL;DR they should belong for the same entry

troyraen

Thanks again! I have a mixture of standardization (eg, capitalization) and wording suggestions. Overall, we should make the tone consistent between the Euclid descriptions and these ones.

troyraen · 2026-02-03T04:14:53Z

tutorials/spherex/spherex.md

+
+- [Spectral Image Cutouts](spherex_cutouts.md) - Demonstrates how to generate and work with spatial and spectral cutouts.
+
+- [PSF Models](spherex_psf.md) - Explains how SPHEREx point spread function (PSF) information is organized and accessed.


I used a different tone for the Euclid descriptions and we should pick one and be consistent. What are your thoughts? I chose an imperative style for the Euclid ones because my philosophy for this kind of documentation is to tell the reader/user what they'll do.

troyraen · 2026-02-03T04:22:10Z

tutorials/techniques.md

+
+- [LSDB HATS](parquet-catalog-demos/irsa-hats-with-lsdb.md) - Uses LSDB and HATs framework for advanced cross-matching and query strategies
+
+- [SED visualization](firefly/SEDs_in_Firefly.md) - Demonstrates how to explore and interactively visualize SEDs using Firefly on multi-wavelength datasets.


Suggested change

- [SED visualization](firefly/SEDs_in_Firefly.md) - Demonstrates how to explore and interactively visualize SEDs using Firefly on multi-wavelength datasets.

- [SED Visualization](firefly/SEDs_in_Firefly.md) - Demonstrates how to explore and interactively visualize SEDs using Firefly on multi-wavelength datasets.

troyraen · 2026-02-03T04:31:34Z

tutorials/wise.md

+
+
+WISE and NEOWISE data are released publicly through the NASA/IPAC Infrared Science Archive (IRSA), including calibrated images, source catalogs, and single-exposure source tables that together enable multi-epoch photometry, light curves, and motion studies for a wide range of astrophysical and Solar System applications. 
+Successive NEOWISE data releases (with annual updates) provide users with increasingly deep coverage and time-domain information across the infrared sky.


This should be in the past tense since the mission is over and all data releases are out. Maybe also give the end date (Aug 2024) in the paragraph above?

troyraen · 2026-02-03T04:38:40Z

tutorials/wise.md

+
+- [ALLWISE Images](irsa-sia-examples/sia_allwise_atlas.md) - Retrieves coadded images and makes coordinate-based cutouts.
+
+- [ALLWISE Catalog](parquet-catalog-demos/wise-allwise-catalog-demo.md) - Explores mid-infrared source properties by querying, filtering, and working with the catalog.


In my view, this notebook is much more about how to work with our HEALPix-partitioned Parquet catalogs than it is about mid-IR source properties.

troyraen · 2026-02-03T04:40:00Z

tutorials/wise.md

+
+- [NEOWISE visualization](firefly/NEOWISE_light_curve_demo.md) - Visualizes and analyzes light curves of Solar System objects using Firefly.
+
+- [NEOWISE source table](parquet-catalog-demos/neowise-source-table-strategies.md) - Outlines efficient strategies for accessing and handling large tables


Suggested change

- [NEOWISE source table](parquet-catalog-demos/neowise-source-table-strategies.md) - Outlines efficient strategies for accessing and handling large tables

- [NEOWISE Strategies](parquet-catalog-demos/neowise-source-table-strategies.md) - Outlines efficient strategies for accessing and handling this very large Parquet dataset.

troyraen · 2026-02-03T05:09:35Z

tutorials/simulated.md

+# Simulated Data Tutorial Notebooks
+
+IRSA hosts a diverse collection of simulated astronomical datasets spanning multiple missions and science domains; designed to support survey planning, algorithm development, and scientific exploration.
+Because this collection is heterogeneous in coverage, structure, and intended use, simulated products are released with detailed documentation and tailored access methods.


Maybe split this into two sentences? I don't think these were released with detailed documentation and tailored access methods because these are simulated/heterogeneous - IRSA releases most data products with those things.

troyraen · 2026-02-03T05:11:51Z

tutorials/simulated.md

+
+- [Roman HLSS Number Density](roman_simulations/roman_hlss_number_density.md) - Illustrates how to query the catalog and derive galaxy number density.
+
+- [OpenUniverse2024 Roman coadds](openuniversesims/openuniverse2024_roman_simulated_wideareasurvey.md) - Accesses OpenUniverse2024 wide-area simulated survey data.


Suggested change

- [OpenUniverse2024 Roman coadds](openuniversesims/openuniverse2024_roman_simulated_wideareasurvey.md) - Accesses OpenUniverse2024 wide-area simulated survey data.

- [OpenUniverse2024 Roman Coadds](openuniversesims/openuniverse2024_roman_simulated_wideareasurvey.md) - Accesses OpenUniverse2024 wide-area simulated survey data.

tutorials/spherex/spherex.md

troyraen · 2026-02-03T05:28:22Z

tutorials/spherex/spherex.md

+
+SPHEREx IRSA data releases include weekly [Quick Release spectral image products](https://caltech-ipac.github.io/spherex-archive-documentation/spherex-data-products/) (multi-extension FITS files containing calibrated near-infrared surface brightness, variance, flags, modeled backgrounds, PSFs, and wavelength WCS) along with ancillary calibration and metadata files such as gain matrices, dark current maps, solid angle pixel maps, and detailed spectral WCS products for each detector.
+
+- [Data Overview](spherex_intro.md) - Introduces available data including how to identify which files and formats are appropriate for different use cases.


This doesn't seem to describe the notebook. All of the data products are in the same FITS file and the user doesn't get a choice about formats.

Co-authored-by: Troy Raen <raen@ipac.caltech.edu>

jkrick self-assigned this Jan 22, 2026

jkrick changed the title ~~adding shorttitle and landing page for spherex~~ adding short_title and landing page for spherex Jan 22, 2026

bsipocz reviewed Jan 22, 2026

View reviewed changes

bsipocz added the html rendering / skip testing Rendering related issues/PRs. Skips tests in PRs. label Jan 22, 2026

jkrick changed the title ~~adding short_title and landing page for spherex~~ adding short_title and landing pages for everything other than Euclid Jan 22, 2026

troyraen reviewed Jan 23, 2026

View reviewed changes

tutorials/techniques.md Outdated Show resolved Hide resolved

toc.yml Show resolved Hide resolved

jkrick and others added 9 commits January 26, 2026 13:57

adding shorttitle and landing page for spherex

a225ef2

Revert tutorials to ignore short_title changes

9e246be

adding wise landing page

c0ff292

adding landing page for simulated data

4380ac4

adding landing page for techniques and tools

bde38e5

ruthlessly reducing length of notebook description sentences in prepa…

aa557d1

…ration for cards

updating toc.yml to have landing pages and reduce layers

8ad5379

fixing paths in landing pages

9719b26

TMP: REMOVE BEFORE MERGE

84b4357

jkrick force-pushed the toc_update branch from 808257b to 84b4357 Compare January 26, 2026 18:57

jkrick added 2 commits January 26, 2026 14:07

updating toc with titles

c65b080

applying Troys review comment

117f9c0

jkrick commented Jan 26, 2026

View reviewed changes

tutorials/techniques.md Outdated Show resolved Hide resolved

jkrick marked this pull request as ready for review January 26, 2026 19:21

fixing spherex links with underscores

842f31a

bsipocz requested changes Jan 26, 2026

View reviewed changes

jkrick changed the title ~~adding short_title and landing pages for everything other than Euclid~~ adding titles to TOC and landing pages for everything other than Euclid Jan 26, 2026

adding spherex titles to toc

07f42d9

bsipocz reviewed Jan 26, 2026

View reviewed changes

toc.yml Outdated Show resolved Hide resolved

toc.yml Outdated Show resolved Hide resolved

toc.yml Outdated Show resolved Hide resolved

toc.yml Outdated Show resolved Hide resolved

tutorials/wise.md Outdated Show resolved Hide resolved

Fix syntax

3f5fc44

removing short titles from landing pages

ac28408

troyraen self-requested a review January 27, 2026 19:20

This was referenced Jan 28, 2026

Do we want a HATS section in TOC under Techniques and Tools for all HATS related notebooks? #227

Open

consider removing the bottom pointers on the rendered pages #228

Closed

jaladh-singhal mentioned this pull request Jan 30, 2026

Auto-generate gallery style cards on index pages #233

Open

troyraen reviewed Feb 3, 2026

View reviewed changes

This was referenced Feb 3, 2026

Add landing pages for Spitzer, 2MASS, and COSMOS #235

Open

Make links to individual sections #152

Open

jkrick and others added 2 commits February 5, 2026 16:22

Apply suggestion from @troyraen

e2bc1d8

Co-authored-by: Troy Raen <raen@ipac.caltech.edu>

Apply suggestion from @troyraen

4ae197d

Co-authored-by: Troy Raen <raen@ipac.caltech.edu>


		- [Spectral Image Cutouts](spherex_cutouts.md) - Demonstrates how to generate and work with spatial and spectral cutouts.

		- [PSF Models](spherex_psf.md) - Explains how SPHEREx point spread function (PSF) information is organized and accessed.


		- [LSDB HATS](parquet-catalog-demos/irsa-hats-with-lsdb.md) - Uses LSDB and HATs framework for advanced cross-matching and query strategies

		- [SED visualization](firefly/SEDs_in_Firefly.md) - Demonstrates how to explore and interactively visualize SEDs using Firefly on multi-wavelength datasets.



		WISE and NEOWISE data are released publicly through the NASA/IPAC Infrared Science Archive (IRSA), including calibrated images, source catalogs, and single-exposure source tables that together enable multi-epoch photometry, light curves, and motion studies for a wide range of astrophysical and Solar System applications.
		Successive NEOWISE data releases (with annual updates) provide users with increasingly deep coverage and time-domain information across the infrared sky.


		- [ALLWISE Images](irsa-sia-examples/sia_allwise_atlas.md) - Retrieves coadded images and makes coordinate-based cutouts.

		- [ALLWISE Catalog](parquet-catalog-demos/wise-allwise-catalog-demo.md) - Explores mid-infrared source properties by querying, filtering, and working with the catalog.


		- [NEOWISE visualization](firefly/NEOWISE_light_curve_demo.md) - Visualizes and analyzes light curves of Solar System objects using Firefly.

		- [NEOWISE source table](parquet-catalog-demos/neowise-source-table-strategies.md) - Outlines efficient strategies for accessing and handling large tables

	- [NEOWISE source table](parquet-catalog-demos/neowise-source-table-strategies.md) - Outlines efficient strategies for accessing and handling large tables
	- [NEOWISE Strategies](parquet-catalog-demos/neowise-source-table-strategies.md) - Outlines efficient strategies for accessing and handling this very large Parquet dataset.


		- [Roman HLSS Number Density](roman_simulations/roman_hlss_number_density.md) - Illustrates how to query the catalog and derive galaxy number density.

		- [OpenUniverse2024 Roman coadds](openuniversesims/openuniverse2024_roman_simulated_wideareasurvey.md) - Accesses OpenUniverse2024 wide-area simulated survey data.


		SPHEREx IRSA data releases include weekly [Quick Release spectral image products](https://caltech-ipac.github.io/spherex-archive-documentation/spherex-data-products/) (multi-extension FITS files containing calibrated near-infrared surface brightness, variance, flags, modeled backgrounds, PSFs, and wavelength WCS) along with ancillary calibration and metadata files such as gain matrices, dark current maps, solid angle pixel maps, and detailed spectral WCS products for each detector.

		- [Data Overview](spherex_intro.md) - Introduces available data including how to identify which files and formats are appropriate for different use cases.

Conversation

jkrick commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jkrick commented Jan 22, 2026

Uh oh!

jkrick commented Jan 22, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bsipocz commented Jan 22, 2026

Uh oh!

jkrick commented Jan 22, 2026

Uh oh!

bsipocz commented Jan 22, 2026

Uh oh!

bsipocz commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jkrick commented Jan 22, 2026

Uh oh!

bsipocz commented Jan 22, 2026

Uh oh!

troyraen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bsipocz commented Jan 26, 2026

Uh oh!

jkrick left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

jkrick commented Jan 26, 2026

Uh oh!

bsipocz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bsipocz commented Jan 26, 2026

Uh oh!

troyraen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

jkrick commented Jan 22, 2026 •

edited

Loading

bsipocz commented Jan 22, 2026 •

edited

Loading