Usage#
The superblockify
package works out of the box, meaning no further downloads are
necessary. Maps are downloaded from the OpenStreetMap API and population data is
downloaded from the GHSL-POP 2023
dataset. Only those map tiles that are needed are cached in the data/ghsl
folder.
The following example uses superblockify
to partition the street network of Scheveningen, a district of The Hague, using the
ResidentialPartitioner
class.
After partitioning, the results will be saved to a GeoPackage file (.gpkg
) that can be opened and
edited with a GIS software like QGIS.
Import#
First, import the package. By convention, superblockify
is shortened as sb
:
import superblockify as sb
2024-04-23 08:20:43,191 | INFO | __init__.py:11 | superblockify version 1.0.0rc8
Configuration#
There are several options to configure superblockify
, see the API Reference.
Common ones are, for example, to log at the “debug” level:
sb.config.set_log_level("DEBUG")
If you already have the GHSL raster file, you can skip its download. In this case, if you already have the file GHS_POP_E2025_GLOBE_R2023A_54009_100_V1_0.tif
inside the data/ghsl
folder, you could set the FULL_RASTER
parameter to NONE
:
sb.config.Config.FULL_RASTER = None
Initialization#
For this example we will use the ResidentialPartitioner
class.
It is a class that partitions a city into superblocks based on the residential street tags in OpenStreetMap.
First, initialize the partitioner with the city name and a search string.
part = sb.ResidentialPartitioner(
name="Scheveningen_test",
city_name="Scheveningen",
search_str="Scheveningen, NL",
unit="time", # "time", "distance", any other edge attribute, or None to count edges
)
2024-04-23 08:20:43,204 | DEBUG | base.py:816 | Finding graph with search string Scheveningen, NL
2024-04-23 08:20:44,849 | DEBUG | utils.py:131 | Reprojected boundary to WGS84 - Downloading graph
2024-04-23 08:20:48,051 | DEBUG | utils.py:139 | Downloaded graph - Preprocessing
2024-04-23 08:20:48,240 | INFO | tessellation.py:101 | Calculating edge cells for graph with 2303 edges.
2024-04-23 08:20:48,374 | DEBUG | tessellation.py:106 | Prepared 1550 edge polygons. Next, interpolating points.
2024-04-23 08:20:49,751 | DEBUG | tessellation.py:135 | Prepared 5840 points. Creating Voronoi tessellation.
2024-04-23 08:20:49,784 | DEBUG | tessellation.py:139 | Reconstructing edge cells.
2024-04-23 08:20:50,176 | INFO | tessellation.py:156 | Tessellated 1550 edge cells in 0:00:01.
2024-04-23 08:20:50,470 | INFO | ghsl.py:129 | Using the GHSL raster tiles for the bounding box (314241.71247182746, 6089247.782207976, 318635.0138841436, 6093526.030021869).
2024-04-23 08:20:50,471 | DEBUG | ghsl.py:183 | The bounding box spans the tiles {(3, 19)}.
2024-04-23 08:20:50,472 | DEBUG | ghsl.py:268 | Creating GHSL directory at /home/runner/work/superblockify/superblockify/docs/data/ghsl...
2024-04-23 08:20:50,473 | DEBUG | ghsl.py:279 | Downloading https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_POP_GLOBE_R2023A/GHS_POP_E2025_GLOBE_R2023A_54009_100/V1-0/tiles/GHS_POP_E2025_GLOBE_R2023A_54009_100_V1_0_R3_C19.zip to /home/runner/work/superblockify/superblockify/docs/data/ghsl...
2024-04-23 08:20:52,890 | DEBUG | ghsl.py:291 | Extracting GHS_POP_E2025_GLOBE_R2023A_54009_100_V1_0_R3_C19.tif to /home/runner/work/superblockify/superblockify/docs/data/ghsl...
2024-04-23 08:20:53,122 | DEBUG | approximation.py:351 | Loading GHSL data for window Window(col_off=3552.417124718275, row_off=9064.739699781312, width=43.93301412316123, height=42.78247813892813) from file /home/runner/work/superblockify/superblockify/docs/data/ghsl/GHS_POP_E2025_GLOBE_R2023A_54009_100_V1_0_R3_C19.tif.
2024-04-23 08:20:53,246 | DEBUG | approximation.py:193 | Building STRtree index.
Distributing population over road cells: 0%| | 0/1550 [00:00<?, ?Cells/s]
Distributing population over road cells: 3100Cells [00:00, 13422.45Cells/s]
Distributing population over road cells: 3100Cells [00:00, 13382.22Cells/s]
2024-04-23 08:20:53,572 | DEBUG | approximation.py:66 | Adding population and area to edges.
2024-04-23 08:20:53,895 | DEBUG | utils.py:180 | Preprocessing done - Saving graph
2024-04-23 08:20:54,035 | DEBUG | utils.py:182 | Saved graph (1 MB) to /home/runner/work/superblockify/superblockify/docs/data/graphs/Scheveningen.graphml
2024-04-23 08:20:54,035 | DEBUG | base.py:824 | Saving graph to /home/runner/work/superblockify/superblockify/docs/data/graphs/Scheveningen.graphml
2024-04-23 08:20:54,312 | DEBUG | utils.py:342 | Removed 10 dead ends.
2024-04-23 08:20:54,328 | INFO | utils.py:264 | Highway counts (type, count, proportion):
count proportion
highway
residential 1831 0.799214
secondary 253 0.110432
tertiary 113 0.049323
living_street 63 0.027499
unclassified 24 0.010476
[unclassified, residential] 5 0.002182
secondary_link 1 0.000436
trunk 1 0.000436
2024-04-23 08:20:54,330 | DEBUG | utils.py:270 | Dtype counts (type, count, proportion):
count proportion
highway
<class 'str'> 2286 0.997818
<class 'list'> 5 0.002182
2024-04-23 08:20:54,332 | INFO | utils.py:294 | Graph stats:
0
Number of nodes 996
Number of edges 2303
Average degree 4.624498
Circuity average 1.039461
Street orientation order 0.077229
Date created 2024-04-23 08:20:47
Projection EPSG:32631
Area by OSM boundary (m²) 14543826.589252
2024-04-23 08:20:54,333 | INFO | base.py:186 | Initialized Scheveningen_test(ResidentialPartitioner) with 986 nodes and 2291 edges.
This will download the map of Scheveningen, preprocess it, output some statistics
and store it in the data/graphs
folder for later use.
Any other partitioner for Scheveningen, given the same city_name
, will use the same
preprocessed, locally stored map.
Population tiles are cached in the data/ghsl
folder (if not using the full raster).
If you want to select a different city, find the corresponding search string (search_str
) at https://nominatim.openstreetmap.org/. The smaller the place, the quicker the partitioning. For large places sufficient memory is required.
Partitioning#
Next, we will show the quickest way to partition the city and calculate the metrics all in one go.
part.run(
calculate_metrics=True, # set to False if you are not interested in metrics
make_plots=True, # set to False if you are not interested in plots
replace_max_speeds=False, # set to true to overwrite the OSM speed limits
# -> with 15 km/h inside Superblocks and 50 km/h outside
# If the approach has specific parameters, you can set them here
)
2024-04-23 08:20:54,337 | DEBUG | streettype.py:30 | Writing residential attribute to graph.
2024-04-23 08:20:54,345 | DEBUG | attribute.py:51 | Found 397 edges with attribute residential == 1, find LCC of them.
2024-04-23 08:20:54,504 | DEBUG | base.py:512 | Making components from sparsified graph, rest graph has 727 nodes and 1609 edges.
2024-04-23 08:20:54,662 | DEBUG | checks.py:46 | Checking if the sparsified graph of Scheveningen_test is connected.
2024-04-23 08:20:54,664 | DEBUG | checks.py:56 | Checking if each subgraph of Scheveningen_test is connected.
2024-04-23 08:20:54,670 | DEBUG | checks.py:62 | Checking if each node and edge of Scheveningen_test is contained in exactly one subgraph and not the sparsified graph.
2024-04-23 08:20:54,713 | DEBUG | checks.py:71 | Checking if each subgraph of Scheveningen_test is connected to the sparsified graph.
2024-04-23 08:20:54,721 | DEBUG | checks.py:79 | Checking if representative nodes of Scheveningen_test are contained in their subgraph.
2024-04-23 08:20:54,737 | INFO | checks.py:87 | The partitioning Scheveningen_test is valid.
2024-04-23 08:20:54,737 | INFO | plot.py:42 | Plotting partitions graph for Scheveningen_test with attribute `residential`
2024-04-23 08:20:54,738 | DEBUG | attribute.py:183 | Given minmax_val for edge attribute residential: (0, 25)
2024-04-23 08:20:54,803 | DEBUG | plot.py:179 | Unique colors in the edge colormap hsv (len 2305): {(1.0, 0.0, 0.0, 1.0), (1.0, 0.2316178786767022, 0.0, 1.0)}
/home/runner/micromamba/envs/sb_env/lib/python3.12/site-packages/osmnx/plot.py:248: UserWarning: *c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*. Please use the *color* keyword-argument or provide a 2D array with a single row if you intend to specify the same RGB or RGBA value for all points.
ax.scatter(
2024-04-23 08:20:55,120 | DEBUG | plot.py:676 | Saving plot () to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_partition_graph.pdf
2024-04-23 08:20:55,310 | DEBUG | plot.py:278 | Using components for plotting.
2024-04-23 08:20:55,318 | DEBUG | attribute.py:183 | Given minmax_val for edge attribute residential: (0, 25)
2024-04-23 08:20:55,318 | DEBUG | plot.py:537 | Plotting component/partition sizes for Scheveningen_test.
2024-04-23 08:20:55,325 | DEBUG | plot.py:676 | Saving plot (Component size of Scheveningen_test) to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_subgraph_component_size.pdf
2024-04-23 08:20:55,710 | DEBUG | plot.py:178 | Using components for plotting.
2024-04-23 08:20:55,711 | INFO | plot.py:196 | Plotting component size rank for Scheveningen_test with attribute `residential`
2024-04-23 08:20:55,811 | DEBUG | plot.py:676 | Saving plot (Component size rank for Scheveningen_test with attribute `residential`) to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_component_rank_size.pdf
2024-04-23 08:20:56,151 | INFO | plot.py:93 | Plotting component graph for Scheveningen_test with attribute `residential`
2024-04-23 08:20:56,204 | DEBUG | plot.py:179 | Unique colors in the edge colormap <matplotlib.colors.LinearSegmentedColormap object at 0x7fb51a779730> (len 2305): {(1.0, 0.3563285278014729, 0.0, 1.0), (0.41420858919315884, 0.0, 0.9964607646437041, 1.0), (0.0, 0.07681612419693695, 0.9947319877630777, 1.0), (0.3278615842663417, 0.9974138929324614, 0.0, 1.0), (0.0, 0.2630991955680465, 0.8440989495935659, 1.0), (0.7405812349888883, 1.0, 0.0, 1.0), (0.3755346982386072, 0.0, 1.0, 1.0), (1.0, 0.16639256895259236, 0.0, 1.0), (1.0, 0.9226162592817655, 0.0, 1.0), (0.926659814410898, 1.0, 0.0, 1.0), (1.0, 0.0, 0.3386295673944123, 1.0), (0, 0, 0, 1), (1.0, 0.7874868509602921, 0.0, 1.0), (1.0, 0.0, 0.0027103909193387558, 1.0), (1.0, 0.7538523108169379, 0.0, 1.0), (1.0, 0.5840534401839677, 0.0, 1.0), (0.2110501599549267, 0.0, 1.0, 1.0), (0.0, 0.3045949202697976, 0.8037184462545544, 1.0), (1.0, 0.0, 0.0, 1.0), (0.5072944378668851, 1.0, 0.0, 1.0), (0.0, 0.4945090516356058, 0.5882647969296434, 1.0), (0.6003387154472675, 0.0, 0.8467000890771819, 1.0), (0.023504734034601293, 0.7101892569291322, 0.2761832844617287, 1.0), (0.8318034462616026, 0.0, 0.5916961439886985, 1.0), (0.1403236815547061, 0.8609830483906624, 0.0, 1.0)}
2024-04-23 08:20:56,396 | DEBUG | plot.py:676 | Saving plot () to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_component_graph.pdf
2024-04-23 08:20:56,578 | INFO | base.py:337 | Calculating metrics for Scheveningen_test
2024-04-23 08:20:56,604 | DEBUG | distances.py:115 | Downcasting indices to int32.
2024-04-23 08:20:56,604 | DEBUG | distances.py:124 | Downcasting indptr to int32.
2024-04-23 08:20:56,743 | DEBUG | distances.py:140 | All-pairs shortest path lengths for graph with 994 nodes and 2305 edges calculated in 0:00:00.138845.
2024-04-23 08:20:57,209 | DEBUG | measures.py:412 | Calculated betweenness centrality in 0:00:00.183566 seconds.
2024-04-23 08:20:57,224 | DEBUG | metric.py:290 | Coverage (length): 0.8274147306431157
2024-04-23 08:20:57,224 | DEBUG | distances.py:394 | Preparing to calculate partitioning distance matrix.
2024-04-23 08:20:57,242 | DEBUG | distances.py:424 | Checked that the partitions do not overlap node-wise.
2024-04-23 08:20:57,247 | DEBUG | distances.py:523 | Preparing semipermeable graphs.
2024-04-23 08:20:57,254 | DEBUG | distances.py:558 | Calculating 2 semipermeable distance matrices.
2024-04-23 08:20:57,279 | DEBUG | distances.py:567 | Finished calculating the first distance matrix in 0:00:00.024599. Starting the second.
2024-04-23 08:20:57,321 | DEBUG | distances.py:578 | Finished calculating distance matrices in 0:00:00.066498. Combining the results.
2024-04-23 08:20:57,326 | DEBUG | distances.py:605 | Prepared distance matrices
2024-04-23 08:20:57,515 | DEBUG | distances.py:707 | Finished filling up paths in 0:00:00.189743. Used 25 tensorized and 0 vectorized steps.
2024-04-23 08:20:57,968 | DEBUG | measures.py:412 | Calculated betweenness centrality in 0:00:00.167778 seconds.
2024-04-23 08:20:57,973 | DEBUG | metric.py:441 | Directness SN: 0.974095380776169
2024-04-23 08:20:57,977 | DEBUG | metric.py:448 | Global efficiency NS: 0.9882237609941489
2024-04-23 08:20:58,156 | DEBUG | plot.py:676 | Saving plot ($d_S(i, j)$) to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_distance_matrices.pdf
2024-04-23 08:20:58,156 | DEBUG | plot.py:688 | Not using tight_layout because one of the axes has no subplotspec.
2024-04-23 08:20:58,596 | DEBUG | plot.py:676 | Saving plot ($d_S(i, j)$) to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_distance_matrices_pairwise_relative_difference.pdf
2024-04-23 08:20:58,597 | DEBUG | plot.py:688 | Not using tight_layout because one of the axes has no subplotspec.
2024-04-23 08:20:58,957 | DEBUG | plot.py:676 | Saving plot (Scheveningen_test - Metric (s) | $\frac{d_{S}(i, j)}{d_{N}(i, j)}$) to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_relative_difference_SN.pdf
2024-04-23 08:20:59,431 | DEBUG | attribute.py:183 | Given minmax_val for edge attribute rel_increase_comp: (1, 1.284262801334693)
2024-04-23 08:20:59,492 | DEBUG | plot.py:179 | Unique colors in the edge colormap RdYlGn_r (len 2305): {(0.31818531334102285, 0.7013456362937333, 0.3677816224529028, 1.0), (0.043983083429450225, 0.489042675893887, 0.2579777008842753, 1.0), (0.8918877354863515, 0.954479046520569, 0.6010765090349867, 1.0), (0.9985390234525182, 0.9547097270280661, 0.6730488273740869, 1.0), (0.6460592079969244, 0.8488273740868897, 0.41514801999231066, 1.0), (0.7568627450980393, 0.8966551326412918, 0.48419838523644754, 1.0), (0.7700884275278739, 0.11810841983852365, 0.15148019992310652, 1.0), (0.08396770472895042, 0.5628604382929643, 0.2964244521337947, 1.0), (0.05597846981930028, 0.5111880046136101, 0.2695117262591311, 1.0), (0.9912341407151096, 0.9963091118800461, 0.7370242214532872, 1.0), (0.5771626297577857, 0.8186851211072665, 0.40761245674740487, 1.0), (0.9988465974625145, 0.9642445213379469, 0.6890426758938869, 1.0), (0, 0, 0, 1), (0.6470588235294118, 0.0, 0.14901960784313725, 1.0), (0.8685121107266437, 0.9446366782006921, 0.5690888119953865, 1.0), (0.8117647058823532, 0.9203383314109959, 0.5197231833910035, 1.0), (0.7647058823529413, 0.9000384467512496, 0.4892733564013842, 1.0), (0.7254901960784316, 0.883121876201461, 0.46389850057670134, 1.0), (0.11949250288350644, 0.6046136101499424, 0.31810841983852367, 1.0), (0.9970780469050365, 0.9987697039600154, 0.7450211457131872, 1.0), (0.051980007689350266, 0.5038062283737025, 0.2656670511341792, 1.0), (0.0199923106497501, 0.4447520184544406, 0.23490965013456364, 1.0), (0.9994617454825068, 0.9833141099577085, 0.7210303729334872, 1.0), (0.08796616685890045, 0.570242214532872, 0.30026912725874666, 1.0), (0.9327950788158401, 0.9717031910803537, 0.6570549788542869, 1.0), (0.8568242983467899, 0.9397154940407536, 0.5530949634755865, 1.0)}
/home/runner/micromamba/envs/sb_env/lib/python3.12/site-packages/osmnx/plot.py:248: UserWarning: *c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*. Please use the *color* keyword-argument or provide a 2D array with a single row if you intend to specify the same RGB or RGBA value for all points.
ax.scatter(
2024-04-23 08:21:00,945 | DEBUG | plot.py:676 | Saving plot () to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_component_wise_travel_increase.pdf
2024-04-23 08:21:01,156 | DEBUG | attribute.py:183 | Given minmax_val for edge attribute rel_increase: (1, 1.2948224903186965)
2024-04-23 08:21:01,194 | DEBUG | plot.py:179 | Unique colors in the edge colormap RdYlGn_r (len 2305): {(0.43936947327950804, 0.75840061514802, 0.39254133025759325, 1.0), (0.0599769319492503, 0.5185697808535179, 0.27335640138408307, 1.0), (0.01199538638985006, 0.42998846597462514, 0.22722029988465975, 1.0), (0.16624375240292205, 0.6273740868896579, 0.32979623221837756, 1.0), (0.043983083429450225, 0.489042675893887, 0.2579777008842753, 1.0), (0.031987697039600166, 0.4668973471741638, 0.24644367550941948, 1.0), (0.9956170703575548, 0.855363321799308, 0.5257208765859285, 1.0), (0.9999231064975009, 0.9976163014225298, 0.7450211457131872, 1.0), (0.27143406382160723, 0.6785851595540178, 0.35609381007304886, 1.0), (0.5870049980776626, 0.8229911572472126, 0.40868896578239144, 1.0), (0.9962322183775472, 0.88319876970396, 0.5530949634755863, 1.0), (0.0799692425990004, 0.5554786620530565, 0.29257977700884275, 1.0), (0.3298731257208767, 0.7070357554786622, 0.3707035755478662, 1.0), (0.25974625144175334, 0.6728950403690889, 0.3531718569780854, 1.0), (0.5377931564782777, 0.8014609765474818, 0.4033064206074587, 1.0), (0.177931564782776, 0.6330642060745868, 0.3327181853133411, 1.0), (0.7019607843137257, 0.8729719338715879, 0.44867358708189164, 1.0), (0.972087658592849, 0.5374086889657824, 0.31349480968858134, 1.0), (0.2129950019223377, 0.6501345636293734, 0.34148404459823145, 1.0), (0.13118031526336033, 0.6103037293348712, 0.32103037293348713, 1.0), (0.9993079584775086, 0.9785467128027683, 0.7130334486735873, 1.0), (0.10780469050365253, 0.5989234909650135, 0.3151864667435602, 1.0), (0.827450980392157, 0.9271049596309113, 0.5298731257208766, 1.0), (0.8117647058823532, 0.9203383314109959, 0.5197231833910035, 1.0), (0.9734717416378317, 0.5474048442906574, 0.31810841983852367, 1.0), (0.6862745098039217, 0.8662053056516724, 0.43852364475201855, 1.0), (0.976239907727797, 0.5673971549404075, 0.3273356401384083, 1.0), (0.07597078046905038, 0.5480968858131489, 0.28873510188389084, 1.0), (0.9951557093425606, 0.8322952710495963, 0.5063437139561706, 1.0), (0.45905420991926194, 0.7670126874279124, 0.39469434832756634, 1.0), (0.6705882352941178, 0.8594386774317571, 0.4283737024221454, 1.0), (0.09196462898885047, 0.5776239907727797, 0.3041138023836986, 1.0), (0.28312187620146106, 0.6842752787389466, 0.3590157631680123, 1.0), (0.9965397923875433, 0.8927335640138409, 0.5690888119953863, 1.0), (0.3649365628604385, 0.7241061130334487, 0.3794694348327567, 1.0), (0.9479430988081508, 0.4086889657823914, 0.25413302575932334, 1.0), (0.449211841599385, 0.7627066512879662, 0.3936178392925798, 1.0), (0.9937716262975779, 0.7630911188004613, 0.4482122260668973, 1.0), (0.5673202614379087, 0.8143790849673203, 0.40653594771241836, 1.0), (0.30649750096116896, 0.6956555171088044, 0.3648596693579393, 1.0), (0.9942329873125721, 0.7861591695501731, 0.4675893886966551, 1.0), (0.24805843906189942, 0.66720492118416, 0.35024990388312194, 1.0), (0.8860438292964246, 0.9520184544405998, 0.5930795847750866, 1.0), (0.15455594002306816, 0.621683967704729, 0.3268742791234141, 1.0), (0.9610149942329873, 0.457439446366782, 0.2765859284890427, 1.0), (0.06397539407920033, 0.5259515570934257, 0.277201076509035, 1.0), (0.3532487504805846, 0.71841599384852, 0.3765474817377932, 1.0), (0.0, 0.40784313725490196, 0.21568627450980393, 1.0), (0.9678585159554017, 0.9864667435601692, 0.7050365244136869, 1.0), (0.9971549404075356, 0.9118031526336025, 0.6010765090349866, 1.0), (0.6470588235294118, 0.0, 0.14901960784313725, 1.0), (0.41968473663975403, 0.7497885428681277, 0.39038831218762016, 1.0), (0.8685121107266437, 0.9446366782006921, 0.5690888119953865, 1.0), (0.007996924259900041, 0.42260668973471743, 0.22337562475970782, 1.0), (0.8039215686274511, 0.9169550173010381, 0.5146482122260669, 1.0), (0.02399077277970012, 0.45213379469434833, 0.2387543252595156, 1.0), (0.7647058823529413, 0.9000384467512496, 0.4892733564013842, 1.0), (0.9269511726259132, 0.9692425990003845, 0.6490580545943868, 1.0), (0.9973087274125336, 0.9165705497885428, 0.6090734332948866, 1.0), (0.5082660515186468, 0.7885428681276433, 0.4000768935024991, 1.0), (0.5279507881584008, 0.7971549404075357, 0.4022299115724722, 1.0), (0.20130718954248378, 0.6444444444444445, 0.338562091503268, 1.0), (0.11949250288350644, 0.6046136101499424, 0.31810841983852367, 1.0), (0.7254901960784316, 0.883121876201461, 0.46389850057670134, 1.0), (0.9651672433679355, 0.48742791234140714, 0.29042675893886966, 1.0), (0.8626682045367168, 0.9421760861207229, 0.5610918877354865, 1.0), (0.9970011534025375, 0.9070357554786621, 0.5930795847750865, 1.0), (0.051980007689350266, 0.5038062283737025, 0.2656670511341792, 1.0), (0.0199923106497501, 0.4447520184544406, 0.23490965013456364, 1.0), (0.9994617454825068, 0.9833141099577085, 0.7210303729334872, 1.0), (0.08796616685890045, 0.570242214532872, 0.30026912725874666, 1.0), (0.07197231833910037, 0.5407151095732411, 0.2848904267589389, 1.0), (0.9983852364475202, 0.9499423298731258, 0.6650519031141868, 1.0), (0.9327950788158401, 0.9717031910803537, 0.6570549788542869, 1.0), (0.5968473663975395, 0.8272971933871588, 0.40976547481737796, 1.0), (0.8568242983467899, 0.9397154940407536, 0.5530949634755865, 1.0), (0.47873894655901594, 0.7756247597078048, 0.3968473663975394, 1.0), (0.015993848519800083, 0.43737024221453286, 0.2310649750096117, 1.0), (0.6362168396770475, 0.8445213379469435, 0.41407151095732414, 1.0), (0.5181084198385238, 0.7928489042675895, 0.4011534025374856, 1.0), (0.9979238754325259, 0.9356401384083045, 0.6410611303344868, 1.0), (0.04798154555940024, 0.4964244521337947, 0.26182237600922725, 1.0), (0.819607843137255, 0.9237216455209535, 0.52479815455594, 1.0), (0.7098039215686276, 0.8763552479815456, 0.45374855824682825, 1.0), (0.003998462129950021, 0.41522491349480967, 0.21953094963475586, 1.0), (0.14286812764321424, 0.6159938485198001, 0.32395232602845064, 1.0), (0.9795463283352557, 0.9913879277201076, 0.7210303729334872, 1.0), (0.6627450980392158, 0.8560553633217993, 0.42329873125720885, 1.0), (0.3415609381007307, 0.712725874663591, 0.3736255286428297, 1.0), (0.9985390234525182, 0.9547097270280661, 0.6730488273740869, 1.0), (0.9444828911956941, 0.9766243752402922, 0.673048827374087, 1.0), (0.0399846212995002, 0.48166089965397924, 0.25413302575932334, 1.0), (0.9845444059976932, 0.6273740868896578, 0.3550173010380623, 1.0), (0.08396770472895042, 0.5628604382929643, 0.2964244521337947, 1.0), (0.05597846981930028, 0.5111880046136101, 0.2695117262591311, 1.0), (0.6941176470588235, 0.8695886197616302, 0.44359861591695504, 1.0), (0.6263744713571704, 0.8402153018069973, 0.41299500192233757, 1.0), (0.7725490196078433, 0.9034217608612073, 0.49434832756632074, 1.0), (0.5574778931180316, 0.8100730488273742, 0.4054594386774318, 1.0), (0.9928489042675894, 0.7169550173010381, 0.40945790080738176, 1.0), (0.5771626297577857, 0.8186851211072665, 0.40761245674740487, 1.0), (0.7803921568627452, 0.906805074971165, 0.4994232987312573, 1.0), (0.9988465974625145, 0.9642445213379469, 0.6890426758938869, 1.0), (0.09596309111880048, 0.5850057670126875, 0.3079584775086505, 1.0), (0.4885813148788929, 0.7799307958477509, 0.397923875432526, 1.0), (0.18961937716262978, 0.6387543252595156, 0.3356401384083045, 1.0), (0.6066897347174165, 0.831603229527105, 0.41084198385236453, 1.0), (0.03598615916955018, 0.4742791234140715, 0.2502883506343714, 1.0), (0.9679354094579009, 0.5074202229911575, 0.2996539792387545, 1.0), (0.37662437524029235, 0.7297962322183776, 0.38239138792772015, 1.0), (0.6165321030372936, 0.8359092656670511, 0.41191849288735105, 1.0), (0.46889657823913894, 0.7713187235678586, 0.3957708573625529, 1.0), (0.09996155324875051, 0.5923875432525952, 0.3118031526336025, 1.0), (0.4295271049596311, 0.7540945790080739, 0.39146482122260673, 1.0), (0.9859284890426759, 0.6373702422145329, 0.3596309111880046, 1.0), (0.7960784313725492, 0.9135717031910804, 0.5095732410611304, 1.0), (0.40984236831987714, 0.7454825067281815, 0.38931180315263364, 1.0), (0.9996155324875048, 0.988081507112649, 0.7290272971933871, 1.0), (0.9737024221453288, 0.9889273356401385, 0.7130334486735871, 1.0), (0.9966935793925413, 0.8975009611687812, 0.5770857362552864, 1.0), (0.31818531334102285, 0.7013456362937333, 0.3677816224529028, 1.0), (0.8918877354863515, 0.954479046520569, 0.6010765090349867, 1.0), (0.7176470588235295, 0.8797385620915034, 0.4588235294117648, 1.0), (0.23637062668204542, 0.6615148019992311, 0.3473279507881584, 1.0), (0.6549019607843138, 0.8526720492118416, 0.4182237600922723, 1.0), (0.9211072664359862, 0.9667820069204153, 0.6410611303344868, 1.0), (0.6460592079969244, 0.8488273740868897, 0.41514801999231066, 1.0), (0.7568627450980393, 0.8966551326412918, 0.48419838523644754, 1.0), (0.4984236831987698, 0.7842368319876971, 0.3990003844675125, 1.0), (0.9301038062283737, 0.3711649365628604, 0.23690888119953865, 1.0), (0.843137254901961, 0.9338715878508267, 0.5400230680507498, 1.0), (0.9873125720876587, 0.647366397539408, 0.36424452133794694, 1.0), (0.9707035755478662, 0.5274125336409073, 0.308881199538639, 1.0), (0.06797385620915035, 0.5333333333333333, 0.28104575163398693, 1.0), (0.3883121876201463, 0.7354863514033065, 0.38531334102268366, 1.0), (0.958246828143022, 0.43744713571703187, 0.267358708189158, 1.0), (0.9925413302575933, 0.7015763168012303, 0.3965397923875432, 1.0), (0.9561707035755479, 0.9815455594002307, 0.689042675893887, 1.0), (0.9968473663975395, 0.9022683583237218, 0.5850826605151866, 1.0), (0.2246828143021916, 0.6558246828143023, 0.34440599769319497, 1.0), (0.9817762399077278, 0.6073817762399077, 0.34579008073817763, 1.0), (0.4000000000000002, 0.7411764705882354, 0.38823529411764707, 1.0), (0.5476355247981546, 0.8057670126874279, 0.4043829296424452, 1.0), (0.7411764705882354, 0.8898885044213765, 0.4740484429065745, 1.0), (0.9094194540561323, 0.9618608227604768, 0.6250672818146867, 1.0), (0.9957708573625529, 0.8630526720492119, 0.5321799307958477, 1.0), (0.9886966551326413, 0.657362552864283, 0.36885813148788926, 1.0), (0.02798923490965014, 0.45951557093425605, 0.24259900038446752, 1.0), (0.29480968858131507, 0.6899653979238756, 0.36193771626297583, 1.0), (0.9997693194925029, 0.9928489042675894, 0.7370242214532872, 1.0), (0.950326797385621, 0.9790849673202615, 0.681045751633987, 1.0)}
/home/runner/micromamba/envs/sb_env/lib/python3.12/site-packages/osmnx/plot.py:248: UserWarning: *c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*. Please use the *color* keyword-argument or provide a 2D array with a single row if you intend to specify the same RGB or RGBA value for all points.
ax.scatter(
2024-04-23 08:21:01,454 | DEBUG | plot.py:676 | Saving plot () to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test_relative_increase_on_graph.pdf
2024-04-23 08:21:01,664 | DEBUG | base.py:344 | Metrics for Scheveningen_test: coverage: 0.8274147306431157; directness: SN: 0.974095380776169; global_efficiency: NS: 0.9882237609941489; high_bc_clustering: 0.5714705619783681; high_bc_anisotropy: 3.1915020523837776; distance_matrix: S: [[ 0. 108.9 10.7 ... 126.9 226.6 224.1]
[108.8 0. 98.1 ... 116.1 215.8 213.3]
[ 10.7 98.2 0. ... 116.2 215.9 213.4]
...
[128.3 117.6 117.6 ... 0. 115.7 118.3]
[227.8 217.1 217.1 ... 120.2 0. 2.6]
[227.5 216.8 216.8 ... 117.6 2.5 0. ]], N: [[ 0. 108.90000153 10.7 ... 126.9 226.6
224.1 ]
[108.80000305 0. 98.09999847 ... 116.1 215.8
213.3 ]
[ 10.7 98.19999695 0. ... 116.2 215.9
213.4 ]
...
[128.3 117.6 117.6 ... 0. 115.7
118.3 ]
[227.8 217.1 217.1 ... 120.2 0.
2.6 ]
[230.3 219.6 219.6 ... 117.6 2.5
0. ]]; predecessor_matrix: S: [[-9999 91 0 ... 804 977 973]
[ 2 -9999 77 ... 804 977 973]
[ 2 91 -9999 ... 804 977 973]
...
[ 2 91 77 ... -9999 977 986]
[ 2 91 77 ... 804 -9999 986]
[ 2 91 77 ... 804 977 -9999]], N: [[-9999 91 0 ... 804 977 973]
[ 2 -9999 77 ... 804 977 973]
[ 2 91 -9999 ... 804 977 973]
...
[ 2 91 77 ... -9999 977 986]
[ 2 91 77 ... 804 -9999 986]
[ 2 91 77 ... 804 977 -9999]]; unit: time;
/home/runner/micromamba/envs/sb_env/lib/python3.12/site-packages/osmnx/convert.py:381: FutureWarning: <class 'geopandas.array.GeometryArray'>._reduce will require a `keepdims` parameter in the future
dupes = edges[mask].dropna(subset=["geometry"])
/home/runner/micromamba/envs/sb_env/lib/python3.12/site-packages/osmnx/convert.py:381: FutureWarning: <class 'geopandas.array.GeometryArray'>._reduce will require a `keepdims` parameter in the future
dupes = edges[mask].dropna(subset=["geometry"])
/home/runner/micromamba/envs/sb_env/lib/python3.12/site-packages/osmnx/convert.py:381: FutureWarning: <class 'geopandas.array.GeometryArray'>._reduce will require a `keepdims` parameter in the future
dupes = edges[mask].dropna(subset=["geometry"])
As you see, this partitions the city and shows the relative increase of the distance metric.
All shown plots are saved as pdf to the data/results/Scheveningen_test
folder.
At any time, is possible to save and load a partitioner object to continue the work later.
# part.save()
# part.load("Scheveningen_test")
The most illustrative and interactive way to view the results is to save them to a geopackage file. This file can be opened in QGIS and edited further.
sb.save_to_gpkg(part, save_path=None)
2024-04-23 08:21:02,374 | INFO | utils.py:101 | Using components attribute to save Superblocks to geodatapackage /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test.gpkg
2024-04-23 08:21:02,444 | INFO | utils.py:147 | Node attributes: Index(['y', 'x', 'street_count', 'lon', 'lat', 'node_betweenness_normal',
'node_betweenness_length', 'node_betweenness_linear',
'node_betweenness_normal_restricted',
'node_betweenness_length_restricted',
'node_betweenness_linear_restricted', 'representative_node_name',
'split', 'geometry'],
dtype='object')
2024-04-23 08:21:02,445 | INFO | utils.py:148 | Edge attributes: Index(['osmid', 'highway', 'length', 'bearing', 'speed_kph', 'travel_time',
'population', 'area', 'cell_id', 'residential',
'travel_time_restricted', 'component_name', 'edge_betweenness_normal',
'edge_betweenness_length', 'edge_betweenness_linear',
'edge_betweenness_normal_restricted',
'edge_betweenness_length_restricted',
'edge_betweenness_linear_restricted', 'rel_increase',
'rel_increase_comp', 'classification', 'geometry'],
dtype='object')
2024-04-23 08:21:02,446 | INFO | utils.py:149 | Superblock attributes: Index(['classification', 'value', 'm', 'n', 'length_total', 'ignore',
'representative_node_id', 'mean_edge_betweenness_normal',
'mean_edge_betweenness_length', 'mean_edge_betweenness_linear',
'mean_edge_betweenness_normal_restricted',
'mean_edge_betweenness_length_restricted',
'mean_edge_betweenness_linear_restricted',
'change_mean_edge_betweenness_normal',
'change_mean_edge_betweenness_length',
'change_mean_edge_betweenness_linear', 'population', 'area',
'population_density', 'k_avg', 'edge_length_total', 'edge_length_avg',
'streets_per_node_avg', 'streets_per_node_counts',
'streets_per_node_proportions', 'intersection_count',
'street_length_total', 'street_segment_count', 'street_length_avg',
'circuity_avg', 'self_loop_proportion', 'node_density_km',
'intersection_density_km', 'edge_density_km', 'street_density_km',
'street_orientation_order', 'representative_node_point'],
dtype='object')
2024-04-23 08:21:02,446 | INFO | utils.py:150 | Graph meta attributes: Index(['created_date', 'created_with', 'crs', 'simplified', 'edge_population',
'boundary_crs', 'boundary', 'area', 'n', 'm', 'k_avg',
'edge_length_total', 'edge_length_avg', 'streets_per_node_avg',
'streets_per_node_counts', 'streets_per_node_proportions',
'intersection_count', 'street_length_total', 'street_segment_count',
'street_length_avg', 'circuity_avg', 'self_loop_proportion',
'node_density_km', 'intersection_density_km', 'edge_density_km',
'street_density_km', 'street_orientation_order'],
dtype='object')
2024-04-23 08:21:02,447 | DEBUG | utils.py:162 | Converting column osmid of type <class 'str'> to str.
2024-04-23 08:21:02,448 | DEBUG | utils.py:162 | Converting column highway of type <class 'str'> to str.
2024-04-23 08:21:02,449 | DEBUG | utils.py:162 | Converting column classification of type <class 'str'> to str.
2024-04-23 08:21:02,450 | DEBUG | utils.py:162 | Converting column classification of type <class 'str'> to str.
2024-04-23 08:21:02,451 | DEBUG | utils.py:162 | Converting column representative_node_id of type <class 'str'> to str.
2024-04-23 08:21:02,452 | DEBUG | utils.py:162 | Converting column streets_per_node_counts of type <class 'str'> to str.
2024-04-23 08:21:02,453 | DEBUG | utils.py:162 | Converting column streets_per_node_proportions of type <class 'str'> to str.
2024-04-23 08:21:02,454 | DEBUG | utils.py:162 | Converting column created_date of type <class 'str'> to str.
2024-04-23 08:21:02,454 | DEBUG | utils.py:162 | Converting column created_with of type <class 'str'> to str.
2024-04-23 08:21:02,455 | DEBUG | utils.py:162 | Converting column crs of type <class 'str'> to str.
2024-04-23 08:21:02,456 | DEBUG | utils.py:162 | Converting column boundary_crs of type <class 'str'> to str.
2024-04-23 08:21:02,573 | INFO | utils.py:170 | Saved 994 nodes to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test.gpkg
2024-04-23 08:21:02,840 | INFO | utils.py:172 | Saved 2305 edges to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test.gpkg
2024-04-23 08:21:02,886 | INFO | utils.py:174 | Saved 25 Superblocks to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test.gpkg
2024-04-23 08:21:02,907 | INFO | utils.py:176 | Saved graph meta to /home/runner/work/superblockify/superblockify/docs/data/results/Scheveningen_test/Scheveningen_test.gpkg
This will save the partitioning results to data/results/Scheveningen_test/{city_name}.gpkg
.
If you calculated the metrics before, they will be available in the layers, for each
Superblock. This includes more metrics than shown in the plots earlier.
The name of the components is saved into a classification
edge
attribute. The sparse graph is saved with the value “SPARSE” into the
classification
edge attribute.
To learn more about the inner workings and background of the package, please see the next Reference section. Otherwise, you can also check out the API documentation.
FAQ#
Can I export the plots to another format?#
Yes, you can export the plots to any format supported by matplotlib.
Just change the PLOT_SUFFIX
attribute
in the Config
class to the desired format.
sb.config.Config.PLOT_SUFFIX = "png" # or "svg", "pdf", etc.
The downloaded city is too big/small/not the right city, can I change this?#
The deciding string for the area to download is the search_str
.
Finding a fitting OSM area is via the Nominatim API.
If you want to see your area before downloading, use
the Nominatim Search.
It helps to be more specific, e.g. “Scheveningen, The Hague, Netherlands”
instead of just “Scheveningen”.
Otherwise, OSM relations IDs, e.g. R13751467
, can be used.
To re-download the map, pass reload_graph=True
when initializing the partitioner.
The Superblocks look too big/small/random when using the ResidentialPartitioner
, why is that?#
The ResidentialPartitioner
uses the residential street tags to find the
Superblocks.
The variation in OSM data quality and street tagging practices can be reflected when using this approach.
The BetweennessPartitioner
instead does not rely on OSM tags but uses the betweenness
centrality - a topological property of the street network. Try this approach if the Superblocks from the ResidentialPartitioner
are not satisfactory .
- superblockify.partitioning.approaches.betweenness.BetweennessPartitioner(name='unnamed', city_name=None, search_str=None, unit='time', graph=None, reload_graph=False, max_nodes=20000)[source]
Partitioner using betweenness centrality of nodes and edges.
Set sparsified graph from edges or nodes with high betweenness centrality.
- superblockify.partitioning.approaches.betweenness.BetweennessPartitioner.write_attribute()
Determine edges with high betweenness centrality for sparsified graph.
Edges with high betweenness centrality are used to construct the sparsified graph.
The high percentile is determined through ranking all edges by their betweenness centrality and taking the top percentile. The percentile is determined by the percentile parameter.
- Parameters:
- percentilefloat, optional
The percentile to use for determining the high betweenness centrality edges, by default 90.0
- scalingstr, optional
The type of betweenness to use, can be normal, length, or linear, by default normal
- max_rangeint, optional
The range to use for calculating the betweenness centrality, by default None, which uses the whole graph. Its unit is meters.
- **kwargs
Additional keyword arguments. calculate_metrics_before takes the make_plots parameter.
- Raises:
- ValueError
If scaling is not normal, length, or linear.
- ValueError
If percentile is not between, 0.0 and 100.0.
Pass the kwargs from the write_attribute method to the BetweennessPartitioner
to
set the parameters for the partitioning, like
BetweennessPartitioner(..., percentile=85.0, scaling="normal", max_range=None)
.
My country has another maximum speed limit, can I change this?#
When calculating the metrics and using replace_max_speeds=True
,
the maximum speed limits are set to 15 km/h inside the Superblocks
and 50 km/h outside of them. If you want to change these values, you can do so
by setting the V_MAX_LTN
and V_MAX_SPARSE
attributes in the Config
class.
sb.config.Config.V_MAX_LTN = 30 # km/h
sb.config.Config.V_MAX_SPARSE = 60 # km/h
Some streets I know are not being used in the partitioning, why is that?#
When downloading the map from OpenStreetMap, we use a specific network filter, which should include the car network.
sb.config.Config.NETWORK_FILTER
'["highway"]["area"!~"yes"]["access"!~"private"]["highway"!~"abandoned|bridleway|bus_guideway|busway|construction|corridor|cycleway|elevator|escalator|footway|path|pedestrian|planned|platform|proposed|raceway|service|steps|track"]["motor_vehicle"!~"no"]["motorcar"!~"no"]["service"!~"alley|driveway|emergency_access|parking|parking_aisle|private"]'
Some process is taking too long or suddenly stops, what can I do?#
If there are warnings or logs that indicate a problem, they might point to the issue.
Be aware that, when analyzing a large city, superblockify needs sufficient resources.
If it runs out of memory, some processes might stop abruptly without warning.
To combat this, you can either try to find a search_str
with a smaller area or
set the MAX_NODES
attribute in the Config
class to a lower value.
When initializing a partitioner, the street network is cut off at this number of nodes,
including the most central nodes. By default, this is set to 20,000.
For further settings, see the other attributes in the Config
class.
- superblockify.config.Config()[source]
Configuration class for superblockify.
- Attributes:
- WORK_DIR
The working directory of the package. This is used to store the graphs and results in subdirectories of this directory. By default, this is the current working directory when the package is imported. This is only used to define the following directories.
- GRAPH_DIR
The directory where the graphs are stored.
- RESULTS_DIR
The directory where the results are stored.
- GHSL_DIR
The directory where the GHSL population data is stored when downloaded.
- V_MAX_LTN
The maximum speed in km/h for the restricted calculation of travel times.
- V_MAX_SPARSE
The maximum speed in km/h for the restricted calculation of travel times for the sparsified graph.
- NETWORK_FILTER
The filter used to filter the OSM data for the graph. This is a string that is passed to the
osmnx.graph_from_place()
function.- CLUSTERING_PERCENTILE
The percentile used to determine the betweenness centrality threshold for the spatial clustering and anisotropy nodes.
- NUM_BINS
The number of bins used for the histograms in the entropy calculation.
- FULL_RASTER
The path and filename of the full GHSL raster. If None, tiles of the needed area are downloaded from the JRC FTP server and stored in the GHSL_DIR directory. <https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_POP_GLOBE_R2023A/GHS_POP_E2025_GLOBE_R2023A_54009_100/V1-0/GHS_POP_E2025_GLOBE_R2023A_54009_100_V1_0.zip>
- DOWNLOAD_TIMEOUT
The timeout in seconds for downloading the GHSL raster tiles.
- logger
The logger for this module. This is used to log information, warnings and errors throughout the package.
- TEST_DATA_PATH
The path to the test data directory.
- HIDE_PLOTS
Whether to hide the plots in the tests.
- PLACES_GENERAL
A list of tuples of the form
(name, place)
wherename
is the name of the place andplace
is the place string that is passed to thesuperblockify.utils.load_graph_from_place()
function.- PLACES_SMALL
Same as
PLACES_GENERAL
but for places of which the graph is small enough to be used in the tests.- PLACES_100_CITIES
100 cities from Boeing et al. (2019) <https://doi.org/10.1007/s41109-019-0189-1> A dictionary of the form
{name: place}
wherename
is the name of the place, andplace
is a dictionary of various attributes. One of them is thequery
attribute which is the place string or a list of place strings. Find the extensive list in the./cities.yml
file.- PLACES_GERMANY
List of cities in Germany by population. All cities with more than 100,000 inhabitants are included. Data from the German Federal Statistical Office.
- PLOT_SUFFIX
The format of the plots. Can be
"png"
,"jpg"
,"pdf"
,"svg"
, etc. Matplotlib uses the Pillow library to save the plots, so all formats supported by Pillow are supported by Matplotlib. <https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html>- MAX_NODES
The maximum number of nodes in the graph. If the graph has more nodes, it is reduced. See
superblockify.partitioning.utils.reduce_graph()
.
If you run into any other issues, feel free to look into the API documentation,
Source Code,
activate debug logs sb.config.set_log_level("DEBUG")
or finally open a new issue.