Planet Python
Last update: January 14, 2024 04:43 AM UTC
January 13, 2024
Python People
Julian Sequeira - Pybites, Australia, Mindset, and Teaching New Programmers
Julian Sequeira is a cofounder of Pybites.
He's a Python coach, a podcaster, a career mindset advocate, and is learning guitar.
Topics include:
- Learning guitar
- Vacationing in Canada
- Pybites
- Splitting finances with Bob
- Building a community and a team
- Coaching
- Conscious positivity
- Australia is full of animals that want to kill you. Except kangaroos.
- Teaching Python to non-technical people
The Complete pytest Course
- Level up your testing skills and save time during coding and maintenance.
- Check out courses.pythontest.com
Test and Code
212: Canon TDD - by Kent Beck
In 2002, Kent Beck released a book called "Test Driven Development by Example".
In December of 2023, Kent wrote an article called "Canon TDD".
With Kent's permission, this episode contains the full content of the article.
Brian's commentary is saved for a followup episode.
Links:
The Complete pytest Course
- Level up your testing skills and save time during coding and maintenance.
- Check out courses.pythontest.com
TechBeamers Python
Top 50 Python Data Structure Exercises (List, Set, Dictionary, and Tuple)
Here are 50 Python Data Structure exercises covering List, Set, Dictionary, and Tuple operations. These are excellent exercises for any beginner learning Python. In the following exercises, you’ll engage in a series of hands-on challenges that traverse the landscape of Lists, Sets, Dictionaries, and Tuples, as well as more advanced concepts like list comprehension. Each […]
The post Top 50 Python Data Structure Exercises (List, Set, Dictionary, and Tuple) appeared first on TechBeamers.
Top 45 Python Exercises on Loops, Conditions, and Range() Function
Here are 45 Python exercises on loops (for, while), if-else statements, and the range() function, along with their solutions. Each exercise comes with a brief description of the problem and a solution that utilizes the mentioned constructs. 45 Python Exercises on Loops, Conditions, and Range() Function If you already have learned the basics of Python […]
The post Top 45 Python Exercises on Loops, Conditions, and Range() Function appeared first on TechBeamers.
Python Engineering at Microsoft
Data Science Day Announcement and Call for Speaker Proposals
We are thrilled to announce Python Data Science Day will be taking place March 14th, 2024; a “PyDay” on Pi Day: 3.14
. If you’re a Python developer, entrepreneur, data scientist, student, or researcher working on projects from hobbyist and start up to enterprise level, you’ll find solutions to modernize your data pipelines and answer complex queries with data.
We’re looking forward to your content submission!
Submit a session or a lightning talk proposal to join the list of amazing speakers for Data Science Day. The CFP opened January 11th, 2024 and closes January 25th, 2024 AOE (Anywhere on Earth). Full schedule will be announced early mid February.
There are two types of submissions:
Session (25 minutes, pre-recorded or live)
Sessions are delivered by no more than two people and can cover high level programming topics. Many times these talks are accompanied by slides, demonstrations or blog posts. This could cover a programming story, how it was approached and the solution; or it could dive deeper into a particular topic or using a suite of features.
Lightning Talk (5 to 7 minutes, pre-recorded)
A Lightning talk is a very short presentation. It is a great introduction to public speaking or a great way to present something short and sweet. These talks typically prioritize a singular idea in a digestible way that inspires further learning.
Good submissions will…
- discuss a
hot or
cool tool, product, or skill. - include accessible content with slides, Notebook, repository, tutorial, or blog post.
- create a thoughtful, cohesive story for a Data Science audience.
Special Guest Speakers on Data Science Day
You could be among the speakers on Python Data Science Day.
| Sarah Kaiser, PhD | Soojin Choi | You! |
|---|---|---|
![]() |
![]() |
|
| Python Cloud Developer Advocate | Jupyter Notebook Product Manager | Speaker at Python Data Science Day |
Submit your CFP
More ways to engage with all the Data Science fun:
- Join the Microsoft Fabric Global AI Hack Together on February 15th to March 4th, 2024. Fabric is an end-to-end AI-powered analytics platform that unites your data and services, including data science and data lakes. Register for the event to participate in live streams every week and solve real-world problems with guidance and a community.
- Read our 14 Days of Python Data Science series where for fourteen days leading up to Python Data Science Day, we will drop cool articles and recipes for using Data Science on Microsoft tools. Link coming soon!
- Check out the Data Science Cloud Skills Challenge if you want to go through some self-paced learning! This challenge is active until April 15th, 2024.
- Join us on Discord at https://aka.ms/python-discord
More Data Science at Microsoft…
The post Data Science Day Announcement and Call for Speaker Proposals appeared first on Python.
January 12, 2024
Real Python
The Real Python Podcast – Episode #187: Serializing Data With Python & Underscore Naming Conventions
Do you need to transfer an extensive data collection for a science project? What's the best way to send executable code over the wire for distributed processing? What are the different ways to serialize data in Python? Christopher Trudeau is back on the show this week, bringing another batch of PyCoder's Weekly articles and projects.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Python Software Foundation
EU’s Cyber Resilience Act Passes with Wins for Open Source
Back in April, we wrote to the community about our concerns for the future of the open source ecosystem generally and CPython and PyPI specifically if the European Cyber Resilience Act (CRA) were to pass in the form that had been shared. At the time, we were worried that in the course of providing software for anyone to use, analyze or change that the PSF and/or the Python community might become legally responsible for security issues in the products that are built with the code components that we are providing for free. We asked for increased clarity, specifically:
The good news is that CRA text* changed a lot between the time the open source community – including the PSF – started expressing our concerns and the Act’s final text which was cemented on December 1st. That text introduces the idea of an “open source steward.”
Furthermore, the final text demonstrates a crisper understanding of how open source software works and the value it provides to the overall ecosystem of software development.
So are we totally done paying attention to European legislation? Ah, while it would be nice for the Python community to be able to cross a few things off our to-do list, that’s not quite how it works. Firstly, the concept of an “open source steward” is a brand new idea in European law. So, we will be monitoring the conversation as this new concept is implemented or interacts with other bits of European law to make sure that the understanding continues to reflect the intent and the realities of open source development. Secondly, there are some other pieces of legislation in the works that may also impact the Python ecosystem so we will be watching the Product Liability Directive and keeping up with the discussion around standard-essential patents to make sure that the effects on Python and open source development are intentional (and hopefully benevolent, or at least benign.)
Thank you to Open Forum Europe (OFE) — especially Ciarán O’Riordan – for bringing the FOSS community together to share our thoughts on how the proposed text would affect open source, thinking about how the goals of the proposed act might be achieved without unintentionally creating a chilling effect for open source and communicating those ideas to legislators. OFE’s work to coordinate our efforts certainly made it easier for the PSF’s concerns to be heard and I’m fairly certain it made it easier for legislators to assess and consider impacts to the open source ecosystem when we were able to speak with one voice.
*The entire Regulation is published here, if you want to dive into the text more deeply.
TechBeamers Python
How to Import Another Python File
When working on big Python projects or trying to organize your code better, you need to know how to bring in code from other Python files. This is called “importing,” and it’s a crucial skill for reusing code and keeping things neat. In this guide, we’ll explore different ways to import another Python file into […]
The post How to Import Another Python File appeared first on TechBeamers.
January 11, 2024
Kay Hayen
Nuitka this week #14
Contents
Communication vs. Coding
After GSoC 2019, it seems I dropped off with communication about Nuitka quite a lot, e.g. I stopped “Nuitka This Week”. The reasons are multi-facetted. I think part of the reason is that I was getting busy, part of it clearly also was Corona. But also a more dreadful change in my private life, where the real life Nuitka, my wife, became ill for a long time. Effectively it’s only become really better mid last year.
I think, this caused me to go full into code for Nuitka, and to launch Nuitka commercial, but generally to become more quiet. I have already relaxed this for a bit, e.g. about Python 3.11, I made a bunch of postings.
So this one is a bit general to start off, but I also provide fairly recent details about what I worked on for 2.0 as well.
Nuitka has evolved a lot
From a usability standpoint, ever since I went all in with Nuitka, but also before, the out of the box experience of Nuitka has become ever better. And even 2.0 will take it noticeable further. It’s the premier choice for Python deployment if you want efficiency. Its onefile mode is pretty great already and is continuously getting better.
It’s fair to say that Nuitka was great in 2019. I think in 2023 it became almost amazing for deployment. This is in large part due to working on the Yaml configuration and these things. In 2024 I hope to get it really smooth.
I actually made posts about the Yaml stuff, and I will resume it shortly, basically it allows people to help improve the deployment side of Nuitka, e.g. missing DLLs and data files, hacks needed, etc. for packages, and it’s quickly becoming better and complete.
Community
On the Discord server, I have been in touch with users of Nuitka a lot more. You are welcome to join us on the Discord server for Nuitka community where you can hang out with the developers and ask questions. It’s not intended as an interactive manual. You are supposed to read the docs for yourself first.
I am also now occasionally on the Python Discord server. Mostly when I get summoned to answer questions that my community thinks make sense, and have been awarded the community role there, which is pretty nice. I seem to make new connections there.
Optimization Work
I think, it’s in vain to explain what I did for performance in all that time. Mostly, some technical debts for Python3 were collected, extending to advantage in in speed of Nuitka over pure Python again. The advantage compared on Python2 was not as present, and still is not, on Python3, but for 3.10 it’s pretty good.
The major breakthroughs have not happened. But I will be taking about the plans, these sure are exciting. Lots of things are in place, some are not, but I hope to get there.
Current Evolutions
So things on my mind right now, for one, I guess, 4 plugin changes that I have yet to document in new postings. Two are visible here in this code.
- module-name: 'toga.platform' # checksum: 4db91cac
variables:
setup_code: 'import toga.platform'
declarations:
'toga_backend_module_name': 'toga.platform.get_platform_factory().__name__'
anti-bloat:
- change_function:
'get_platform_factory': "'importlib.import_module(%r)' % get_variable('toga_backend_module_name')"
First, the checksum. Nuitka is going to warn you about checking your user yaml files for correctness in the future. Since it often finds structural problems, very much needed, since yaml is whitespace sensitive, and you never know what it is a list, a dict, etc. but the schema we created, can tell.
Second, variables are a new section, and in fact so new, they are
not even documented. They can be used to query at compile time values
from code. In this case we are using it to get at the backend to use, so
we can tell it at runtime. Otherwise, it’s hidden to Nuitka, and could
e.g. still be subject to a changed decision from environment variables,
something we typically do not want.
For the third and forth thing, we need another example. Torch can use a JIT to speed up some things, with a compilation very similar to what Nuitka does. That however needs a compiler and the source code on the target platform. Not an easy ask for all kinds of deployments. A new feature makes this easier than before.
- module-name: 'torch' # checksum: ada8ede8
parameters:
- 'name': 'enable-jit'
'values': 'value in ("yes", "no")'
options:
checks:
- description: "Torch JIT is disabled by default, make a choice explicit with '--module-parameter=torch-disable-jit=yes|no'"
support_info: 'parameter'
when: 'standalone and get_parameter("disable-jit", None) is None'
import-hacks:
- force-environment-variables:
'PYTORCH_JIT': '0'
when: 'get_parameter("disable-jit", "no" if standalone else "yes") == "yes"'
So, what this does, is to make Nuitka accept parameters. The options part is designed to complain when the default value is used in standalone mode, kind of making the user acknowledge that it’s the intended value. For accelerated mode, we do not disable the JIT, since we can expect to be in the same environment with source code intact.
With get_parameter you get the option value, and can be conditional
on it in the when block. That is the 3rd new thing.
The forth new thing, is the forcing of environment variables. We have so
far done this, including in plugins like tk-inter manually with
post-load-code. The above is the same, effectively doing
os.environ["PYTORCH_JIT"] = "0" if the JIT is to be disabled.
These changes are designed to avoid having to do plugins again.
Historically for toga support, we should have been a new plugin, but
now it’s not, since the Yaml mechanism can cover retrieval of compile
time values from modules. And for torch and the JIT, a plugin would
have been needed to provide the command line control for that decision.
This increased power of the Yaml will make it even less often the case that a plugin must be written. But of course docs will have to be added and maybe more places will need to work with variables to make that true even more often. More on that in the future.
Teasers
Future TWN will speak about Nuitka-Python (our own Python fork with incredible capabilities), about Nuitka-Watch (our way of making sure Nuitka works with PyPI packages and hot-fixes to not regress), about compilation reports as a new feature, Windows AV stuff, onefile improvements, and so on and so on. I got interesting stuff for many weeks. Limiting myself for now or I will never publish this.
Twitter and Mastodon
I should be more active there, although often I fall prey to of not wanting to talk about unfinished things, so actually I do not post there as much.
And lets not forget, having followers make me happy. So do re-tweets. Esp. those, please do them.
Help Wanted
Duplicate implicit target name: “help wanted”.
If you are interested, I am tagging issues help wanted and there is a bunch, and very likely at least one you can help with.
Nuitka definitely needs more people to work on it.
January 10, 2024
TestDriven.io
Django vs. Flask in 2024: Which Framework to Choose
In this article, we'll look at the best use cases for Django and Flask along with what makes them unique, from an educational and development standpoint.
Nikola
Nikola v8.3.0 is out!
On behalf of the Nikola team, I am pleased to announce the immediate availability of Nikola v8.3.0. This release adds support for Python 3.12, with some other features and bugfixes.
Note that Nikola v8.3.0 no longer uses the Yapsy plugin manager, which has been replaced by a custom, minimal manager. The new Nikola Plugin Manager was tested with some typical plugins, but there might be issues if your plugins have an unusual structure or are outdated. Please update your plugins and report any bugs you may encounter.
The Nikola developers would also like to express discontent with Python’s efforts to remove features from the standard library, causing breakage without a solid reason, other than “it’s old”.
What is Nikola?
Nikola is a static site and blog generator, written in Python. It can use Mako and Jinja2 templates, and input in many popular markup formats, such as reStructuredText and Markdown — and can even turn Jupyter Notebooks into blog posts! It also supports image galleries, and is multilingual. Nikola is flexible, and page builds are extremely fast, courtesy of doit (which is rebuilding only what has been changed).
Find out more at the website: https://getnikola.com/
Downloads
Install using pip install Nikola.
Changes
Features
Implement a new plugin manager from scratch to replace Yapsy, which does not work on Python 3.12 due to Python 3.12 carelessly removing parts of the standard library (Issue #3719)
Support for Discourse as comment system (Issue #3689)
Bugfixes
Fix loading of templates from plugins with
__init__.pyfiles (Issue #3725)Fix margins of paragraphs at the end of sections (Issue #3704)
Ignore
.DS_Storefiles in listing indexes (Issue #3698)Fix baguetteBox.js invoking in the base theme (Issue #3687)
Fix development (preview) server
nikola autofor non-root SITE_URL, in particular when URL_TYPE is full_path. (Issue #3715)
For plugin developers
Nikola now requires the .plugin file to contain a [Nikola]
section with a PluginCategory entry set to the name of the plugin
category class. This was already required by plugins.getnikola.com,
but you may have custom plugins that don’t have this set.
Real Python
Python range(): Represent Numerical Ranges
A range is a Python object that represents an interval of integers. Usually, the numbers are consecutive, but you can also specify that you want to space them out. You can create ranges by calling range() with one, two, or three arguments, as the following examples show:
>>> list(range(5))
[0, 1, 2, 3, 4]
>>> list(range(1, 7))
[1, 2, 3, 4, 5, 6]
>>> list(range(1, 20, 2))
[1, 3, 5, 7, 9, 11, 13, 15, 17, 19]
In each example, you use list() to explicitly list the individual elements of each range. You’ll study these examples in more detail soon.
In this tutorial, you’ll learn how you can:
- Create
rangeobjects that represent ranges of consecutive integers - Represent ranges of spaced-out numbers with a fixed step
- Decide when
rangeis a good solution for your use case - Avoid
rangein most loops
A range can sometimes be a powerful tool. However, throughout this tutorial, you’ll also explore alternatives that may work better in some situations. You can click the link below to download the code that you’ll see in this tutorial:
Get Your Code: Click here to download the free sample code that shows you how to represent numerical ranges in Python.
Construct Numerical Ranges
In Python, range() is built in. This means that you can always call range() without doing any preparations first. Calling range() constructs a range object that you can put to use. Later, you’ll see practical examples of how to use range objects.
You can provide range() with one, two, or three integer arguments. This corresponds to three different use cases:
- Ranges counting from zero
- Ranges of consecutive numbers
- Ranges stepping over numbers
You’ll learn how to use each of these next.
Count From Zero
When you call range() with one argument, you create a range that counts from zero and up to, but not including, the number you provided:
>>> range(5)
range(0, 5)
Here, you’ve created a range from zero to five. To see the individual elements in the range, you can use list() to convert the range to a list:
>>> list(range(5))
[0, 1, 2, 3, 4]
Inspecting range(5) shows that it contains the numbers zero, one, two, three, and four. Five itself is not a part of the range. One nice property of these ranges is that the argument, 5 in this case, is the same as the number of elements in the range.
Count From Start to Stop
You can call range() with two arguments. The first value will be the start of the range. As before, the range will count up to, but not include, the second value:
>>> range(1, 7)
range(1, 7)
The representation of a range object just shows you the arguments that you provided, so it’s not super helpful in this case. You can use list() to inspect the individual elements:
>>> list(range(1, 7))
[1, 2, 3, 4, 5, 6]
Observe that range(1, 7) starts at one and includes the consecutive numbers up to six. Seven is the limit of the range and isn’t included. You can calculate the number of elements in a range by subtracting the start value from the end value. In this example, there are 7 - 1 = 6 elements.
Count From Start to Stop While Stepping Over Numbers
Read the full article at https://realpython.com/python-range/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Django Weblog
DSF membership now recognizes a much broader range of contributions to Django
Recently, the DSF made some changes to our bylaws to change the definition of DSF Membership. You can read the legalese of the new language in the meeting minutes for the October 12 board meeting, but here’s the short version: previously, individual membership required contribution of intellectual property (e.g. code or documentation) we’ve changed it so that individual membership now recognizes broader contributions to the DSF’s mission. That still includes code and docs, but now also includes many more activities: organizing a Django event, serving on a Working Group, maintaining a third-party app, moderating Django community spaces, and much more. (Corporate membership hasn’t changed; this just applies to individual membership.)
The DSF’s mission, as described in our bylaws, is:
The Foundation's purposes shall include, but not be limited to, developing and promoting the Django framework for free and open public use among the worldwide web development community, protecting the framework's long-term viability, and advancing the state of the art in web development.
Membership, then, recognizes material contributions to that mission. This is deliberately broad and inclusive: we want to allow as broad a definition of “contribution” as possible – including, critically, contributions to the community as well as code contributions. But we do want those contributions to be “material”: we want to recognize substantial or sustained contributions, not one-offs or “drive-by” contributions.
Because this definition of “material” is somewhat deliberately vague, we’ve prepared an FAQ that outlines several examples of things we believe do and do not qualify someone for membership. Ultimately, though, if you’re not sure: please apply anyway! We generally try to err on the side of saying “yes”.
To join the DSF under these new, more inclusive rules, fill out the application form here. The Board approves new members at its monthly meeting, so you can expect to hear back within about a month.
Read the Docs
Read the Docs newsletter - January 2024
News and updates
We have shipped New improvements to redirects, making our redirects much more powerful and flexible.
We have shipped an updated approach to notifications. Currently there isn’t much UX difference, but as we move forward with this project we will be able to provide more context and control to users.
We continue to work on improving Addons, our new approach to documentation integrations. New documentation and bug fixing continues to happen.
We shipped version 2.0 of our Read the Docs Sphinx Theme, which adds support for new Sphinx releases and drops support for many old versions of Sphinx and Python.
You can always see the latest changes to our platforms in our Read the Docs Changelog.
Upcoming changes
Addons will be made more configurable in our new beta dashboard, starting a trend of moving away from the old dashboard for new features.
Our beta dashboard continues to be tested in public beta, and new functionality for Addons configuration will only be available in that new interface.
We continue to work on some business model changes enabled by the new redirects work, including allowing access to Forced Redirects for more users.
Want to follow along with our development progress? View our full roadmap 📍️
Possible issues
Users need to update their webhooks before January 31, 2024 if they are configured without a secret. All users who need to take action should have received email and site notifications about this.
We are discussing removing support for all VCS systems except Git, as our userbase is heavily biased towards Git users and it will simplify maintenance and development of features. We stopped developing features for Mercurial, Subversion, and Bazaar years ago, and we are considering removing support for them entirely. We will be reaching out to these users to get feedback on this change.
Questions? Comments? Ideas for the next newsletter? Contact us!
TypeThePipe
Plot your GIS data with GeoPandas and Plotnine. A single glance insightful visualization
pre > code.sourceCode { white-space: pre; position: relative; } pre > code.sourceCode > span { display: inline-block; line-height: 1.25; } pre > code.sourceCode > span:empty { height: 1.2em; } .sourceCode { overflow: visible; } code.sourceCode > span { color: inherit; text-decoration: inherit; } div.sourceCode { margin: 1em 0; } pre.sourceCode { margin: 0; } @media screen { div.sourceCode { overflow: auto; } } @media print { pre > code.sourceCode { white-space: pre-wrap; } pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; } } pre.numberSource code { counter-reset: source-line 0; } pre.numberSource code > span { position: relative; left: -4em; counter-increment: source-line; } pre.numberSource code > span > a:first-child::before { content: counter(source-line); position: relative; left: -1em; text-align: right; vertical-align: baseline; border: none; display: inline-block; -webkit-touch-callout: none; -webkit-user-select: none; -khtml-user-select: none; -moz-user-select: none; -ms-user-select: none; user-select: none; padding: 0 4px; width: 4em; color: #aaaaaa; } pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; } div.sourceCode { } @media screen { pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; } } code span.al { color: #ff0000; font-weight: bold; } /* Alert */ code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */ code span.at { color: #7d9029; } /* Attribute */ code span.bn { color: #40a070; } /* BaseN */ code span.bu { color: #008000; } /* BuiltIn */ code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */ code span.ch { color: #4070a0; } /* Char */ code span.cn { color: #880000; } /* Constant */ code span.co { color: #60a0b0; font-style: italic; } /* Comment */ code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */ code span.do { color: #ba2121; font-style: italic; } /* Documentation */ code span.dt { color: #902000; } /* DataType */ code span.dv { color: #40a070; } /* DecVal */ code span.er { color: #ff0000; font-weight: bold; } /* Error */ code span.ex { } /* Extension */ code span.fl { color: #40a070; } /* Float */ code span.fu { color: #06287e; } /* Function */ code span.im { color: #008000; font-weight: bold; } /* Import */ code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */ code span.kw { color: #007020; font-weight: bold; } /* Keyword */ code span.op { color: #666666; } /* Operator */ code span.ot { color: #007020; } /* Other */ code span.pp { color: #bc7a00; } /* Preprocessor */ code span.sc { color: #4070a0; } /* SpecialChar */ code span.ss { color: #bb6688; } /* SpecialString */ code span.st { color: #4070a0; } /* String */ code span.va { color: #19177c; } /* Variable */ code span.vs { color: #4070a0; } /* VerbatimString */ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */

Geographical Data Visualization in Python with GeoPandas
Working with geographical data can often be a bit tricky for the uninitiated. This post aims to shed some light for those who are encountering geo data for the first time and want an easy way to perform initial manipulations and plot it to gain their first insights.
We’re about to dive into a sea of coordinates, shapes, and a whole lot of mapping magic!
GIS data formats
First of all, we want to make a summary about the most used data formats for geo data. These are the unsung heroes in the world of mapping and spatial analysis. Imagine trying to describe the vastness of our planet without them – it’s like trying to paint a masterpiece with just one color. Not happening, right?
First up in our parade of geo data superstars are formats like Shapefile (SHP), the old faithful of the geo world. But you’ll need a complete set of three files that are mandatory to make up a shapefile. The three required files are: .SHP for geometry, .SHX is the shape index position and .DBF is the attribute data.
Developed by ESRI, it’s like the trusty old pickup truck – not the flashiest but gets the job done. Then there’s GeoJSON, the cool kid on the block. It’s all about simplicity and web-friendliness, perfect for those who love to mingle with JavaScript and web mapping.
And let’s not forget about KML (Keyhole Markup Language), the go-to for Google Earth enthusiasts. It’s like having a GPS in your pocket – straightforward and ready to guide you through those virtual globetrotting adventures. Each of these formats has its own quirks and charms, kind of like different types of pasta – some are better suited for hearty meaty sauces, while others are perfect for a light seafood affair.
In short, the world of geo data formats is as diverse and colorful as the world it represents. Whether you’re a GIS guru or a casual map enthusiast, getting to know these formats is like getting the keys to a whole new world of spatial wonders
Yuo can check more info here!
Load SHAP data into Python object
Let’s kick things off by accessing the geo data and geometries. Imagine we are a company with central stations spread across the Spanish territory, responsible for the alerts in their vicinity. Each station acts as a sentinel, vigilantly monitoring and responding to the events unfolding around it. So we will begin with defining the country over which we’re planning to plot the stations and event coordinates.
Now, here comes the fun part: we load the Shape data using GeoPandas. It’s like unlocking a treasure chest of geographical wonders! GeoPandas makes it a breeze, turning what could be a complex task into a walk in the park. Imagine GeoPandas as your trusty GPS, guiding you through the intricate world of geo data with ease and precision. So, grab your data, let’s fire up GeoPandas, and watch as those lines of code magically transform into a map full of possibilities!
import geopandas as gp
import pandas as pd
def filter_shp_peninsular_data(df: pd.DataFrame) -> pd.DataFrame:
return df[df["acom_name"] != "Canarias"]
spain_gis_map = gp.read_file('./data/georef-spain-provincia/georef-spain-provincia-millesime.shp')
peninsular_gis_map = filter_shp_peninsular_data(spain_gis_map)We can filter out any layer in the GeoPandas read_file function with the bbox and mask parameters. But let’s keep it simple for the moment and just load and filter out it.
For ease of use, we are going to load and filter also the alerts peninsular data. You can easily apply coordinates filters in order to focus in one specific geography. We will transform our internal data read with Polars to GeoPandas dataframe to show how it could be do, but it is not strictly necessary.
from shapely.geometry import Point
import polars as pl
def filter_df_peninsular_data(df: pd.DataFrame) -> pd.DataFrame:
peninsular_bounds_min = peninsular_gis_map["geometry"].bounds.min()
peninsular_bounds_max = peninsular_gis_map["geometry"].bounds.max()
return df.filter(
(
pl.col("x")>=peninsular_bounds_min["minx"]
) & (
pl.col("x")<=peninsular_bounds_max["maxx"]
) & (
pl.col("y")>=peninsular_bounds_min["miny"]
) & (
pl.col("y")<=peninsular_bounds_max["maxy"]
)
)
alerts_geo_df = pl.read_csv("alers_data.csv")
alerts_geo_df = gp.GeoDataFrame({
"geometry": alerts_geo_df["coord"].map_elements(lambda x: Point(x)),
"alert_solved": alerts_geo_df["alert_solved"].is_not_null(),
"x": alerts_geo_df["x"],
"y": alerts_geo_df["y"],
})
stations_geo_df = pl.read_csv("stations_data.csv")
stations_geo_df = gp.GeoDataFrame({
"geometry": stations_geo_df["coord"].map_elements(lambda x: Point(x)),
"x": stations_geo_df["x"],
"y": stations_geo_df["y"],
})Geopandas plotting with Plotnine
Let’s start with our mission to plot a map sprinkled with stations (in cheerful yellow) and events (in bold red and blue).
Once we have the data both country layers and our project datapoints properly formated as GeoPandas DataFrames
The geom_map function is our stroke of genius, turning geographical data into visual narratives.
(
ggplot()
+ geom_map(peninsular_gis_map, fill=None)
+ geom_map(alerts_geo_df, aes(fill="alert_solved"), size=2)
+ geom_map(stations_geo_df, colour="yellow", size=3)
+ labs(
title="Alerts solved y/n, by location + Stations (yellow)",
caption = "Data from 2023-01-01 to 2023-02-01",
)
)
Geopandas plotting the most representative category by location
Now, we’re not just mapping points; we’re painting a picture of the most representative category by location. Think of it as a data detective story, where each clue (or data point) reveals a part of the bigger picture.
Here’s how we crack the case:
(
ggplot()
+ geom_bin2d(alerts_geo_df, aes(x="x", y="y", fill="alert_solved"), bins = 30)
+ geom_map(stations_geo_df, colour="yellow", size=3)
+ labs(
title="Alerts solved y/n, by location + Stations (yellow)",
caption = "Data from 2023-01-01 to 2023-02-01",
)
)![]()
With geom_bin2d, we’re transforming our map into a vibrant tapestry, showcasing the most contacted categories in a kaleidoscope of colors. Each square on this grid is like a pixel, together weaving the story of our data’s journey across the Spanish landscape. And, of course, our stations, marked in sunny yellow, stand out as beacons in this sea of information.
Stay updated on Python tips
Hopefully, this post has helped familiarize you with GeoPandas, GIS data nd Plotnine in Python.
If you want to stay updated…
#mc_embed_signup{background:#fff; clear:left; font:14px Helvetica,Arial,sans-serif; width:100%;} #mc_embed_signup .button { background-color: #0294A5; /* Green */ color: white; transition-duration: 0.4s; } #mc_embed_signup .button:hover { background-color: #379392 !important; }January 09, 2024
PyCoder’s Weekly
Issue #611 (Jan. 9, 2024)
#611 – JANUARY 9, 2024
View in Browser »
How to Get the Current Time in Python
In this video course, you’ll be getting the current time in Python. You’ll get your hands on a datetime object that represents the current time. You’ll see how to format it according to international standards, and you’ll even check out how computers represent time.
REAL PYTHON course
Using a Markov Chain to Generate Readable Nonsense
Describes a simple Markov chain algorithm to generate reasonable-sounding but utterly nonsensical text, and presents some example outputs as well as a Python implementation in only 20 lines of code.
BEN HOYT
End-to-End Data Science Workflows: ML Workflows With Python
Learn Machine Learning workflow with Python & Posit Team in this insightful video. Starting with an example classification model, through design, development, deployment, & maintenance, learn from our experience with thousands of enterprise customers →
POSIT sponsor
Top Python Libraries of 2023
Tryolab’s 9th annual Top Python libraries list. Loads of LLM stuff this time around, not surprisingly.
DESCOINS & ALFARO
Python GUI Designer “PAGE” 8.0 Released
THEDESIGNATEDGEEK.XYZ • Shared by Greg Walters
Python Jobs
Senior Python Architect and Tech Lead (America)
Python Tutorial Editor (Anywhere)
Articles & Tutorials
Efficient Inequality Joins in Pandas
Joins in Pandas are equality based. However, there are other scenarios that require joins on inequality or a combination of equi and non-equi joins. Usually in Pandas this is executed via a Cartesian join which can be inefficient as well as memory intensive. This article shows an efficient way to handle inequality joins in Pandas.
SAMUEL ORANYELI • Shared by Samuel oranyeli
2023 Real Python Tutorial & Video Course Wrap-Up
Three members of the Real Python team are joining us this week: Kate Finegan, Tappan Moore, and Philipp Acsany. We wanted to share a year-end wrap-up with tutorials, step-by-step projects, code conversations, and video courses that showcase what our team created in 2023.
REAL PYTHON podcast
Advice for New Devs Who’ve Read Other Advice Essays
After reading some programming advice posts, this author decided a lot of them concentrated on the wrong things. Here is his own take. Associated HN discussion.
HILLEL WAYNE
Building a Bootstrap Styled Form in Vanilla Django
When playing with django-crispy-forms, David wondered what it would take to build a Bootstrap 5 form in Django without any third party libraries. This article outlines the work involved.
DAVID SMITH
Where Have You Installed Your Python Packages?
This article introduces virtual environments and what it means to run pip install. It includes information about the sys module and variables it uses to look for your packages.
KE PI
How to Use Python in Excel Natively
How to use Python in Excel natively using libraries like Pandas, NumPy, Matplotlib, Seaborn and more for analysis and spectacular charts. Includes a cheat sheet.
MYNDA TREACY
My Proof-of-Concept Record Type
As part of a conversation as to whether Python should include a struct-like syntax, Brett has created a project. This post outlines the proof-of-concept.
BRETT CANNON
Python Rgonomics
If you’re coming to Python from R, this article outlines some libraries that have an R-like feel, helping you make the transition to Pythonic workflows.
EMILY RIEDERER
10 Things Software Developers Should Learn About Learning
Developers want and need to perpetually learn. Needing to learn and knowing how are two different things though, and this article highlights key ideas to teach yourself how to learn better.
BROWN, HERMANS, & MARGULIEUX
Don’t Use Python’s @property
A quick opinion piece on the over-use of the @property decorator and how it can lead to misunderstandings in your code.
JAMES BENNETT
10 Noteworthy AI Research Papers of 2023
This article is a compilation of 10 noteworthy AI research papers of 2023.
SEBASTIAN RASCHKA
Projects & Code
Events
Building Python Communities Around Python for Kids
January 10 to January 11, 2024
NOKIDBEHIND.ORG
Weekly Real Python Office Hours Q&A (Virtual)
January 10, 2024
REALPYTHON.COM
Python Atlanta
January 12, 2024
MEETUP.COM
PyDelhi User Group Meetup
January 13, 2024
MEETUP.COM
DFW Pythoneers 2nd Saturday Teaching Meeting
January 13, 2024
MEETUP.COM
Inland Empire Python Users Group Monthly Meeting
January 17, 2024
MEETUP.COM
Happy Pythoning!
This was PyCoder’s Weekly Issue #611.
View in Browser »
[ Subscribe to 🐍 PyCoder’s Weekly 💌 – Get the best Python news, articles, and tutorials delivered to your inbox once a week >> Click here to learn more ]
PyCharm
How to Connect Django With MongoDB
One of the most popular web frameworks, Django, has adopted the “batteries included” philosophy. This means that you can build a production-ready application using only the vanilla Django with no extensions or other frameworks. Even the database SQLite is available out of the box. SQLite is great for learning and developing small applications, but has […]
Real Python
Python Basics Exercises: Functions and Loops
As you learned in Python Basics: Functions and Loops, functions serve as the fundamental building blocks in almost every Python program. They’re where the real action happens!
You now know that functions are crucial for breaking down code into smaller, manageable chunks. They enable you to define actions that your program can execute repeatedly throughout your code. Instead of duplicating the same code whenever your program needs to accomplish a particular task, you can simply call the function.
However, there are instances when you need to repeat certain code multiple times in a row. This is where loops become invaluable.
In this Python Basics Exercises video course, you’ll practice:
- Creating user-defined functions
- Implementing
forloops - Getting user input
- Rounding numbers
This video course is part of the Python Basics series, which accompanies Python Basics: A Practical Introduction to Python 3. You can also check out the other Python Basics courses.
Note that you’ll be using IDLE to interact with Python throughout this course.
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Python Bytes
#366 Put It In The Backlog
<strong>Topics covered in this episode:</strong><br> <ul> <li><a href="https://tonybaloney.github.io/posts/python-gets-a-jit.html"><strong>Python 3.13 gets a JIT</strong></a></li> <li><a href="https://fosstodon.org/@basnijholt/111605262871991435">UniDep - Unified Conda and Pip Dependency Management</a></li> <li><a href="https://hynek.me/articles/pull-requests-branch/"><strong>Don’t Start Pull Requests from Your Main Branch</strong></a></li> <li><a href="https://github.com/pomponchik/instld"><strong>instld: The simplest package management</strong></a></li> <li><strong>Extras</strong></li> <li><strong>Joke</strong></li> </ul><a href='https://www.youtube.com/watch?v=PRaTs3PnJvI' style='font-weight: bold;'>Watch on YouTube</a><br> <p><strong>About the show</strong></p> <p>Sponsored by us! Support our work through:</p> <ul> <li>Our <a href="https://training.talkpython.fm/"><strong>courses at Talk Python Training</strong></a></li> <li><a href="https://courses.pythontest.com/p/the-complete-pytest-course"><strong>The Complete pytest Course</strong></a></li> <li><a href="https://www.patreon.com/pythonbytes"><strong>Patreon Supporters</strong></a></li> </ul> <p><strong>Connect with the hosts</strong></p> <ul> <li>Michael: <a href="https://fosstodon.org/@mkennedy"><strong>@mkennedy@fosstodon.org</strong></a></li> <li>Brian: <a href="https://fosstodon.org/@brianokken"><strong>@brianokken@fosstodon.org</strong></a></li> <li>Show: <a href="https://fosstodon.org/@pythonbytes"><strong>@pythonbytes@fosstodon.org</strong></a></li> </ul> <p>Join us on YouTube at <a href="https://pythonbytes.fm/stream/live"><strong>pythonbytes.fm/live</strong></a> to be part of the audience. Usually Tuesdays at 11am PT. Older video versions available there too.</p> <p><strong>Brian #1:</strong> <a href="https://tonybaloney.github.io/posts/python-gets-a-jit.html"><strong>Python 3.13 gets a JIT</strong></a></p> <ul> <li>Anthony Shaw</li> <li>Great article that walks through JIT concepts with a small example as if you were writing a parser in Python instead of C.</li> <li>Covers <ul> <li>What is a JIT?</li> <li>What is a copy-and-patch JIT? and Why?</li> <li>How does the Python JIT work?</li> <li>Is it faster?</li> <li>This is a building block to future improvements</li> </ul></li> </ul> <p><strong>Michael #2:</strong> <a href="https://fosstodon.org/@basnijholt/111605262871991435">UniDep - Unified Conda and Pip Dependency Management</a></p> <ul> <li>🔄 Single requirements.yaml for both #Conda & #Pip.</li> <li>⚙️ Works with pyproject.toml & setup.py.</li> <li>🏢 Perfect for monorepos.</li> <li>🔒 Create consistent conda-lock files for multiple projects.</li> <li>🌍 Platform-specific support.</li> <li>🚀 <code>unidep install</code> for easy setup.</li> <li><a href="https://github.com/basnijholt/unidep">Full source page</a>.</li> </ul> <p><strong>Brian #3:</strong> <a href="https://hynek.me/articles/pull-requests-branch/"><strong>Don’t Start Pull Requests from Your Main Branch</strong></a></p> <ul> <li>Hynek Schlawack</li> <li>When contributing to other users’ repositories, always start a new branch in your fork.</li> <li>Reasons to not use main <ul> <li>Forces you to only have one change in progress</li> <li>Merges will generate conflicts and you can’t pull from that branch anymore. Need to kill the fork and start over</li> <li>If the target repo has branch protection on, then maintainers can’t push to your branch.</li> </ul></li> <li>Hynek also provides a way to fix things if you’ve already started your changes on a main branch fork.</li> </ul> <p><strong>Michael #4:</strong> <a href="https://github.com/pomponchik/instld"><strong>instld: The simplest package management</strong></a></p> <ul> <li>Thanks to this package, it is very easy to manage the lifecycle of packages.</li> <li>⚡ Run your code without installing libraries.</li> <li>⚡ You can use 2 different versions of the same library in the same program.</li> <li>⚡ You can use incompatible libraries in the same project, as well as libraries with incompatible/conflicting dependencies.</li> <li>⚡ It's easy to share written scripts. The script file becomes self-sufficient - the user does not need to install the necessary libraries.</li> <li>⚡ The library does not leave behind "garbage". After the end of the program, no additional files remain in the system.</li> </ul> <p><strong>Extras</strong> </p> <p>Brian: </p> <ul> <li><a href="https://courses.pythontest.com/p/complete-pytest-course">The Complete pytest Course</a> is now actually complete <ul> <li>Although updates will happen when and if necessary as pytest/Python changes.</li> <li>To celebrate, <strong><em>use code 2024 in January for 10% off</em></strong> any pricing option.</li> </ul></li> <li>More episodes of <a href="https://pythonpeople.fm">Python People</a> and <a href="https://podcast.pythontest.com">Python Test</a> on the way now <ul> <li>That course took up a lot of my time in late 2023</li> <li>Just released an <a href="https://pythonpeople.fm/episodes/will-vincent-django-writing-technical-books">episode with Will Vincent</a> and Python Test will have a new episode this week and for the foreseeable future.</li> <li>Let me know if you want to be on Python People or Python Test</li> </ul></li> </ul> <p>Michael:</p> <ul> <li><em>Hatch follow up: Great coverage of Hatch v1.8.0! One small correction: only the binaries for Hatch are signed with the certificate from the PSF.</em> - <strong>Ofek</strong></li> <li><a href="https://status.python.org/incidents/0mld3fml68nd">PyPI new user registration temporarily suspended</a></li> <li>Pagefind and <a href="https://fosstodon.org/@mkennedy/111637520985150159">how I’m using it</a></li> <li><a href="https://www.youtube.com/watch?v=KeegA_uzzSo">Talk Python Live: Data Doodles event</a> coming early Feb</li> <li>New essay: <a href="https://mkennedy.codes/posts/ai-features-a-waste-of-time/">AI Features a Waste of Time?</a></li> </ul> <p><strong>Joke:</strong> <a href="https://workchronicles.com/put-it-in-the-backlog/">Put it in the backlog</a></p>
Python⇒Speed
NumPy 2 is coming: preventing breakage, updating your code
If you’re writing scientific or data science code with Python, there’s a good chance you’re using NumPy, directly or indirectly. Pandas, Scikit-Image, SciPy, Scikit-Learn, AstroPy… these and many other packages depend on NumPy.
NumPy 2 is a new major release, with a release candidate coming out February 1st 2024, and a final release a month or two later. Importantly, it’s backwards incompatible; not in a major way, but enough that some work might be required to upgrade. And that means you need to make sure your application doesn’t break when NumPy 2 comes out.
In this article we’ll cover:
- The different ways the new release might break your application.
- A quick reminder about the importance of pinning packages.
- How to ensure your application doesn’t install NumPy 2 until you’re ready.
- How to easily upgrade your code to support NumPy 2.
Seth Michael Larson
Security Developer-in-Residence Weekly Report #24
Security Developer-in-Residence Weekly Report #24
This critical role would not be possible without funding from the OpenSSF Alpha-Omega project. Massive thank-you to Alpha-Omega for investing in the security of the Python ecosystem!
Welcome to the first weekly report of 2024!
Software Bill-of-Materials for CPython
Continuing from 2023 there will be a focus on Software Bill-of-Materials (SBOMs) for CPython and incremental improvements to the CPython release process as more is automated.
I made a suggestion to release managers to backport SBOM tooling in the CPython repository to all supported release streams in an effort to treat SBOMs more like an additional artifact instead of a new feature of CPython. This would mean SBOMs would be available for previous CPython releases and we won't have to wait until the 3.13.0 stable release in October to make them available for consumption.
Trusted Publisher provenance on PyPI
Last week William Woodruff published a pre-PEP discussion for using Trusted Publisher configurations to bootstrap publish provenance on PyPI. I was involved in reviewing the initial draft, so I'm excited to see the discussion! Some things to highlight that came from the discussion:
- Don't want to start this work concretely until there are more than one Trusted Publisher provider for PyPI. Currently, PyPI only supports GitHub.
- Getting the user interface right on PyPI to not overemphasize what publish provenance implies for consumers or make projects without publish provenance feel "insecure". Having "verified" URLs to the source repository seems like a good place to start?
- Publish provenance isn't build provenance, build provenance requires more than what Trusted Publishers is able to provide on its own.
- Donald's comment on making things better for a common platform (in this case, GitHub and future Trusted Publisher providers) without requiring everyone to switch to that platform.
- Not everyone uses automated deployment workflows, we'll need to design a build integrity mechanism that supports these use-cases. I commented my thoughts on such a system using third-party observations on build reproducibility from a claimed source.
- There are many reasons why folks aren't using Trusted Publishers, even when on GitHub.
Looking forward to helping however I can with this project once it is proposed as a PEP!
Build reproducibility of macOS artifacts
Previously I worked on build reproducibility for CPython source artifacts which are both tarballs. I want to provide build reproducibility to all artifacts that CPython provides including the Windows and macOS binary installers.
Turns out that macOS's Package files (.pkg) use the eXtensible ARchive (XAR) format internally.
This format isn't supported by diffoscope, the tool I've been using for verifying.
I put together a quick bit of functionality in order to diff .pkg files which appears
to work nicely and have submitted it upstream to the diffoscope project.
Next steps for reproducibility would be to apply diffoscope inside an automated macOS build process to shake out any sources of non-determinism and address them.
Software identifiers
Listened to the Open Source Security Podcast (which I recommend) where Josh Bressers and Kurt Seifried discussed software identifiers as they relate to vulnerabilities and Software Bill-of-Materials and more specifically CISA's RFI on software identifiers and OpenSSF's response.
CPE system could work if it was open for others to collaborate. Currently, mostly a closed system. I've also found casually that CPEs tend to work much better for returning CVE matches today compared to Package URLs even though OSV works with Package URLs natively. I suspect tooling will improve in this area as time goes on.
Package URLs (PURLs) are distributed, namespaced, and intrinsic (easily discoverable). Downside is that two completely different Package URLs may reference the same software but different methods of retrieval (which may be relevant!) Sometimes ties software identity to its source code platform which can change (see CPython moving to GitHub).
Package URLs being namespaced also means that they can carve out namespaces that are governed by different standards,
for example the pkg:pypi/... namespace is governed by PEPs for names and versions where pkg:npm/ is governed
by different standards. I think this ability will be critical for software identifiers to model different
ecosystems, ecosystems won't converge to one set of standards so identifying software needs to be able to
model them properly.
Other items
- Attended the OpenSSF Alpha-Omega monthly public meeting. Had some great discussions about "Secure by Design".
- Spent a good chunk of time planning high-level what the first new major projects for 2024 would be, there will be more to share as we approach the start of those projects.
- Triaging multiple reports to the Python Security Response Team.
That's all for this week! 👋 If you're interested in more you can read last week's report.
Thanks for reading! ♡ Did you find this article helpful and want more content like it? Get notified of new posts by subscribing to the RSS feed or the email newsletter.
This work is licensed under CC BY-SA 4.0
January 08, 2024
Django Weblog
DSF Board 2024 Elections – New board members 👋
Following our 2024 DSF Board Election Results, here are quick introductions from our two new board members, Sarah Abderemane and Thibaud Colas, elected for a two-year term for 2024-2025.
Collage: Sarah on the left, smiling, in the Versailles Hall of Mirrors. Thibaud on the right, in a field, looking in the distance with a boy on his shoulders.
Sarah Abderemane
Sarah Abderemane, also known as sabderemane, is a software developer in France. She currently works at Kraken Tech, part of the Octopus Energy group, contributing to solutions to climate change. She works mainly on the backend, but also likes to work on the frontend in her spare time.
She organizes the Django Paris meetup and is an active member of the Django community: she is one of the organizers of the Djangonaut Space program, a member of the Django Accessibility team and maintainer of djangoproject.com.
Outside of open source and work, she is passionate about dance, likes customizing things like mechanical keyboards, and loves to travel to discover new cultures.
Social media and blog:
- Blog: sarahabd.com
- Mastodon: @sabderemane@mastodon.social
- Twitter: @sabderemane_
- LinkedIn: @sarahabd
Thibaud Colas
Thibaud Colas (pronounced /tee-bo/) is a developer based in the UK, working for Torchbox on the Wagtail open source CMS as part of the core team. For Wagtail, Thibaud also contributes to efforts around accessibility, developer relations, as well as participation to programs like Outreachy and Google Summer of Code.
For Django, Thibaud has been involved with organizing and volunteering at events like Django Girls and DjangoCon Europe. More recently, he helped start the accessibility team, and has joined the Djangonaut Space program as a navigator.
Outside of work, Thibaud spends most of his time with two little tornadoes that are 5 and 2 years old. He also enjoys watching sumo and baking macarons. To learn more about Thibaud, check out his personal website thib.me , @thibaud_colas on Twitter/X, @thibaudcolas@fosstodon.org on Mastodon.
Meeting our new board members
Both Sarah and Thibaud are active on the Django Forum, come say hi in their introduction thread!
They will also be taking part in a mini Django contributions sprint and lead discussions on the “Future of Django” as part of Django Cologne’s 50th Meetup.
Anarcat
Last year on this blog
So this blog is now celebrating its 21st birthday (or 20 if you count from zero, or 18 if you want to be pedantic), and I figured I would do this yearly thing of reviewing how that went.
Number of posts
2022 was the official 20th anniversary in any case, and that was one of my best years on record, with 46 posts, surpassed only by the noisy 2005 (62) and matching 2006 (46). 2023, in comparison, was underwhelming: a feeble 11 posts! What happened!
Well, I was busy with other things, mostly away from keyboard, that I will not bore you with here...
The other thing that happened is that the one-liner I used to collect stats was broken (it counted folders and other unrelated files) and wildly overestimated 2022! Turns out I didn't write that much then:
anarc.at$ ls blog | grep '^[0-9][0-9][0-9][0-9].*.md' | se
d s/-.*// | sort | uniq -c | sort -n -k2
57 2005
43 2006
20 2007
20 2008
7 2009
13 2010
16 2011
11 2012
13 2013
5 2014
13 2015
18 2016
29 2017
27 2018
17 2019
18 2020
14 2021
28 2022
10 2023
1 2024
But even that is inaccurate because, in ikiwiki, I can tag any page as being featured on the blog. So we actually need to process the HTML itself because we don't have much better on hand without going through ikiwiki's internals:
anarcat@angela:anarc.at$ curl -sSL https://anarc.at/blog/ | grep 'href="\./' | grep -o 20[0-9][0-9] | sort | uniq -c
56 2005
42 2006
19 2007
18 2008
6 2009
12 2010
15 2011
10 2012
11 2013
3 2014
15 2015
32 2016
50 2017
37 2018
19 2019
19 2020
15 2021
28 2022
13 2023
Which puts the top 10 years at:
$ curl -sSL https://anarc.at/blog/ | grep 'href="\./' | grep -o 20[0-9][0-9] | sort | uniq -c | sort -nr | head -10
56 2005
50 2017
42 2006
37 2018
32 2016
28 2022
19 2020
19 2019
19 2007
18 2008
Anyway. 2023 is certainly not a glorious year in that regard, in any case.
Visitors
In terms of visits, however, we had quite a few hits. According to Goatcounter, I had 122 300 visits in 2023! 2022, in comparison, had 89 363, so that's quite a rise.
What you read
I seem to have hit the Hacker News front page at least twice. I say "seem" because it's actually pretty hard to tell what the HN frontpage actually is on any given day. I had 22k visits on 2023-03-13, in any case, and you can't see me on the front that day. We do see a post of mine on 2023-09-02, all the way down there, which seem to have generated another 10k visits.
In any case, here were the most popular stories for you fine visitors:
Framework 12th gen laptop review: 24k visits, which is surprising for a 13k words article "without images", as some critics have complained. 15k referred by Hacker News. Good reference and time-consuming benchmarks, slowly bit-rotting.
That is, by far, my most popular article ever. A popular article in 2021 or 2022 was around 6k to 9k, so that's a big one. I suspect it will keep getting traffic for a long while.
Calibre replacement considerations: 15k visits, most of which without a referrer. Was actually an old article, but I suspect HN brought it back to light. I keep updating that wiki page regularly when I find new things, but I'm still using Calibre to import ebooks.
Hacking my Kobo Clara HD: is not new but always gathering more and more hits, it had 1800 hits in the first year, 4600 hits last year and now brought 6400 visitors to the blog! Not directly related, but this iFixit battery replacement guide I wrote also seem to be quite popular
Everything else was published before 2023. Replacing Smokeping with Prometheus is still around and Looking at Wayland terminal emulators makes an entry in the top five.
Where you've been
People send less and less private information when they browse the web. The number of visitors without referrers was 41% in 2021, it rose to 44% in 2023. Most of the remaining traffic comes from Google, but Hacker News is now a significant chunk, almost as big as Google.
In 2021, Google represented 23% of my traffic, in 2022, it was down to 15% so 18% is actually a rise from last year, even if it seems much smaller than what I usually think of.
| Ratio | Referrer | Visits |
|---|---|---|
| 18% | 22 098 | |
| 13% | Hacker News | 16 003 |
| 2% | duckduckgo.com | 2 640 |
| 1% | community.frame.work | 1 090 |
| 1% | missing.csail.mit.edu | 918 |
Note that Facebook and Twitter do not appear at all in my referrers.
Where you are
Unsurprisingly, most visits still come from the US:
| Ratio | Country | Visits |
|---|---|---|
| 26% | United States | 32 010 |
| 14% | France | 17 046 |
| 10% | Germany | 11 650 |
| 6% | Canada | 7 425 |
| 5% | United Kingdom | 6 473 |
| 3% | Netherlands | 3 436 |
Those ratios are nearly identical to last year, but quite different from 2021, where Germany and France were more or less reversed.
Back in 2021, I mentioned there was a long tail of countries with at least one visit, with 160 countries listed. I expanded that and there's now 182 countries in that list, almost all of the 193 member states in the UN.
What you were
Chrome's dominance continues to expand, even on readers of this blog, gaining two percentage points from Firefox compared to 2021.
| Ratio | Browser | Visits |
|---|---|---|
| 49% | Firefox | 60 126 |
| 36% | Chrome | 44 052 |
| 14% | Safari | 17 463 |
| 1% | Others | N/A |
It seems like, unfortunately, my Lynx and Haiku users have not visited in the past year. It seems like trying to read those metrics is like figuring out tea leaves...
In terms of operating systems:
| Ratio | OS | Visits |
|---|---|---|
| 28% | Linux | 34 010 |
| 23% | macOS | 28 728 |
| 21% | Windows | 26 303 |
| 17% | Android | 20 614 |
| 10% | iOS | 11 741 |
Again, Linux and Mac are over-represented, and Android and iOS are under-represented.
What is next
I hope to write more next year. I've been thinking about a few posts I could write for work, about how things work behind the scenes at Tor, that could be informative for many people. We run a rather old setup, but things hold up pretty well for what we throw at it, and it's worth sharing that with the world...
So anyway, thanks for coming, faithful reader, and see you in the coming 2024 year...
Real Python
Learn From 2023's Most Popular Python Tutorials and Courses
Python is always getting better, and 2023 brought a number of exciting developments. As part of the Faster CPython project, Python 3.12 is speedier than previous versions. The new release brings improved error messages and more powerful f-strings. You can also enjoy support for the Linux perf profiler, static typing improvements, and changes to subinterpreters.
The Python Software Foundation (PSF) focused on enhancing security in 2023, with the help of inaugural Security Developer-in-Residence Seth Michael Larson. As part of this comprehensive effort, PyPI completed its first security audit. Plus, the year brought new major versions of pandas, Django, and Pydantic. Another exciting development was Mojo, a superset of Python that’s specifically designed for use in machine-learning contexts.
Here at Real Python, we’ve published over a hundred written tutorials and video courses this year, plus weekly podcast episodes. Keep reading to revisit an old favorite or embark on a new learning journey. You can also check out the wrap-up podcast episode.
Join Now: Click here to join the Real Python Newsletter and you'll never miss another Python tutorial, course update, or post.
Practice Your Skills
Practice makes perfect, especially when it comes to programming. This year, the Real Python team expanded the Python Basics learning path with exercises designed to strengthen your Python skills and help you apply your knowledge in real-world scenarios.
In these courses, you can test your knowledge by completing review exercises and challenges before comparing your work to the instructors’ solutions:
- Python Basics Exercises: Numbers and Math
- Python Basics Exercises: Object-Oriented Programming
- Python Basics Exercises: Modules and Packages
By actively engaging with these exercises, you’ll solidify your understanding of Python concepts and build confidence in your programming abilities. Another way to test your knowledge is by taking quizzes, and we added several in 2023. For help along the way, be sure to join Real Python’s community chat and weekly office hours.
Customize Your Coding Environment
Your coding environment plays a crucial role in your productivity as a programmer, so it’s definitely worthwhile to explore your options and make yourself at home. That means picking the perfect editor or IDE for your use case and then tweaking how it works for you.
In these tutorials, you’ll learn how to customize your coding environment to suit your needs:
- Boost Your Coding Productivity With Ptpython
- Choosing the Best Coding Font for Programming
- JupyterLab for an Enhanced Notebook Experience
By exploring these resources, you’ll discover tools and techniques that can enhance your coding experience, making you more efficient and comfortable as you write Python code.
Read the full article at https://realpython.com/popular-python-tutorials-2023/ »
[ Improve Your Python With 🐍 Python Tricks 💌 – Get a short & sweet Python Trick delivered to your inbox every couple of days. >> Click here to learn more and see examples ]
Montreal Python User Group
Montréal-Python 2024 Elections
Dear Pythonistas, after many years of commitment, the current Python Montreal board of directors team has decided to pass the torch. We are now opening the doors to new, passionate members of the Python community to continue to make our organization shine.
Election calendar:
- Application period officially open
- Announcement of applications: January 22, 2024 on our social networks
- Opening of the online voting period from January 22 to 31, 2024
- Physical event to learn more about the candidates and interact with them: January 29, 2024 at MP-103
- Announcement of the new CA: February 1, 2024
If you share our passion for Python and would like to actively contribute to its development in Montréal, we invite you to fill out this application form.


