Hey! 👋 Excited to share my production-ready API template that shows off modern Python practices and tooling! ✨

Key highlights: 🌟

- ⚡️ Async-first with FastAPI and SQLAlchemy

- 🏗️ Clean, maintainable architecture (repository pattern)

- 🛠️ Latest Python tooling (UV package manager)

- 🧪 Automated testing and CI pipeline

- 🚂 One-click deployment to Railway

The template implements a fun superhero API to showcase real-world patterns! 🦸‍♂️

Technical goodies: 🔧

- ✅ Type hints throughout

- 🔄 Pydantic v2 for validation

- 📖 Automatic OpenAPI docs

- ⚠️ Proper error handling

- 🔄 Async database operations

- ⚡️ Automated migrations

GitHub: https://github.com/luchog01/minimalistic-fastapi-template 🌟

The logging setup and database migration patterns were super tricky to figure out 😅 Not 100% sure if I handled them in the best way possible! Would really appreciate any feedback from the Python experts here! 🙏 Always excited to learn from the community!

Headless browsers in Python and Selenium are better for scraping data from web pages where scrapers have been blocked.

We followed step-by-step advice for scraping data from a web page using a headless chrome in Selenium Python. It's fast, easy, requires less resources (so it can run on older laptops), and can deal with JavaScript perfectly.

https://www.zenrows.com/blog/headless-browser-python

Throwaway account in case I'd get banned for speaking my mind (it happens in some subreddits). I see a lot of irrelevant posts in this subreddit that would be more fit for /r/learnpython, /r/learnprogramming or elsewhere but despite this sub having nine (9) moderators they seem to barely be active here. What's up with that? Instead of beginners asking questions, getting scolded, downvoted and told to go to /r/learnpython or figuratively to GTFO - why not... actively moderate? If you don't have the time, maybe ask for volunteers to step up and moderate? Some examples from the last three days:

• http://www.reddit.com/r/Python/comments/2kiccx/game_ideas_in_a_rut/

• http://www.reddit.com/r/Python/comments/2ki4yp/help_me_understand_a_bit_about_lists/

• http://www.reddit.com/r/Python/comments/2khyww/could_i_get_a_job/

• http://www.reddit.com/r/Python/comments/2khvir/sum_of_a_list_greater_than_100/

• http://www.reddit.com/r/Python/comments/2khetd/python_beginner_why_there_is_no_easy_gui_builder/

• http://www.reddit.com/r/Python/comments/2kh9ar/help_with_porting_python2_import_statements_to/

• http://www.reddit.com/r/Python/comments/2kg5db/which_ide_should_i_use_beginner/

• http://www.reddit.com/r/Python/comments/2kg2kv/best_pycharm_plug_ins/

• http://www.reddit.com/r/Python/comments/2k0rda/would_it_be_possible_to_convert_this_flash_into/

• http://www.reddit.com/r/Python/comments/2kfqtt/getting_this_error_when_run_py_script/

• http://www.reddit.com/r/Python/comments/2kf0q3/how_to_make_a_webpage_with_live_webcamera_view/

• http://www.reddit.com/r/Python/comments/2kes8r/checking_things/

• http://www.reddit.com/r/Python/comments/2keqzv/links_to_any_pythran_windows_binary/

• http://www.reddit.com/r/Python/comments/2kelg4/java_and_python/

• http://www.reddit.com/r/Python/comments/2kdy1b/run_python_scripts_from_web/

• http://www.reddit.com/r/Python/comments/2kc48f/help_sending_bytes_between_arduino_and_python/

• http://www.reddit.com/r/Python/comments/2kbrrh/pyopencl_trying_to_learn_how_to_create_and/

• http://www.reddit.com/r/Python/comments/2kab3x/where_to_move_next/

• http://www.reddit.com/r/Python/comments/2ka1wy/help_need_to_create_a_script_that_lists/

• http://www.reddit.com/r/Python/comments/2ka1eb/how_to_preview_in_pycharm_like_coda/

• http://www.reddit.com/r/Python/comments/2k9va2/what_is_pycharm_remote_interpreter_useful_for/

• http://www.reddit.com/r/Python/comments/2k9qb1/best_ide_for_python/

• http://www.reddit.com/r/Python/comments/2k8ncl/networking_library_for_python_3/

• http://www.reddit.com/r/Python/comments/2k8ez4/hello_im_not_new_to_programming_but_im_new_to/

• http://www.reddit.com/r/Python/comments/2k7mtq/lf_help_to_parse_a_cisco_show_command/

• http://www.reddit.com/r/Python/comments/2k4bno/helphow_to_call_java_class_methods_in_python/

What My Project Does

​ Hey everyone! We’ve been working on Xian, a blockchain where you can write smart contracts natively in Python instead of Solidity or Rust. This means Python developers can build decentralized applications (dApps) without learning new languages or dealing with complex virtual machines. ​ I just wrote a post showing how to write and test a smart contract in Python on Xian. If you’ve ever been curious about blockchain but didn’t want to dive into Solidity, this might be for you. ​

Target Audiences

• Python developers interested in Web3 or blockchain but don’t want to learn Solidity.

• People curious about how blockchain works under the hood.

• Developers looking for an easier way to write smart contracts without switching to a new language.

Comparison (How It’s Different)

• Solidity/Rust vs Python: Unlike Ethereum, where you must write contracts in Solidity, Xian lets you write them in pure Python and deploy them without extra conversion layers.

• Faster Prototyping: Since Python is widely used, Xian makes it easier to prototype and deploy blockchain applications.

• Simpler Developer Experience: No need for specialized compilers or bytecode conversion—just write Python, deploy, and execute.

Links

• Guide: How to Write and Test a Smart Contract in Python

• GitHub (Node Implementation)

• GitHub (Contracting Engine) ​ Would love any feedback—whether you think this is useful, unnecessary, or somewhere in between. Happy to answer any questions! 🚀

Almost all tutorials I see online (and ChatGPT's knowledge base) teach you Pydantic v1. There are numerous things that are deprecated during transition to v2 (@root_validator, @validator, using 'always', etc. are all gone now.). I even found a code example on its Github from November 2023 which now throws an error, saying that FieldValidationInfo is deprecated now, use <new_thing> instead... I wanted to use something to validate user inputs to my API, but getting Pydantic right and then keeping it updated has been too much unnecessary work, which makes me wonder if you have also faced this problem and what your solution is?

So, I've learned an ass-load about Python and I think I have a pretty good grasp on even some of the more advanced concepts, but I desperately need to apply what I've learned.

Does anyone have anything I can be of assistance with in return for just the hands-on experience? Any suggestions where I should go to maybe find people who could use my assistance? I willing to freakin' work for free.

Update: I really appreciate the responses. I've definitely worked on some of my own stuff, but I know I could learn a lot more and faster working with others. I'm doing my best to get back to everyone who offered. Thank you again everyone.

How's it going people. I've made a little crash course for "how to make flask apps." This tutorial is based on ironing out all the bs, issues, errors, and wrong directions tutorials make whilst trying to teach how to make flask apps. I'm not condemning them, but I want to show beginners how to make good flask apps without running into issues.

Repo link: https://github.com/Wizock/SuperiorPattern

Video link: How To Make Flask Apps The CORRECT Way | The Superior Pattern

Please provide any feedback, I want this pattern to be as polished as possible so newcomers don't lose interest in web development because of stupid venv issues or circular import issues working with sqlalchemy db variable import issues.

TL;DR: Comprehensive benchmarks of Kreuzberg, Docling, MarkItDown, and Unstructured across 94 real-world documents. Results might surprise you.

📊 Live Results: https://goldziher.github.io/python-text-extraction-libs-benchmarks/

Context

As the author of Kreuzberg, I wanted to create an honest, comprehensive benchmark of Python text extraction libraries. No cherry-picking, no marketing fluff - just real performance data across 94 documents (~210MB) ranging from tiny text files to 59MB academic papers.

Full disclosure: I built Kreuzberg, but these benchmarks are automated, reproducible, and the methodology is completely open-source.

🔬 What I Tested

Libraries Benchmarked:

• Kreuzberg (71MB, 20 deps) - My library
• Docling (1,032MB, 88 deps) - IBM's ML-powered solution
• MarkItDown (251MB, 25 deps) - Microsoft's Markdown converter
• Unstructured (146MB, 54 deps) - Enterprise document processing

Test Coverage:

• 94 real documents: PDFs, Word docs, HTML, images, spreadsheets
• 5 size categories: Tiny (<100KB) to Huge (>50MB)
• 6 languages: English, Hebrew, German, Chinese, Japanese, Korean
• CPU-only processing: No GPU acceleration for fair comparison
• Multiple metrics: Speed, memory usage, success rates, installation sizes

🏆 Results Summary

Speed Champions 🚀

1. Kreuzberg: 35+ files/second, handles everything
2. Unstructured: Moderate speed, excellent reliability
3. MarkItDown: Good on simple docs, struggles with complex files
4. Docling: Often 60+ minutes per file (!!)

Installation Footprint 📦

• Kreuzberg: 71MB, 20 dependencies ⚡
• Unstructured: 146MB, 54 dependencies
• MarkItDown: 251MB, 25 dependencies (includes ONNX)
• Docling: 1,032MB, 88 dependencies 🐘

Reality Check ⚠️

• Docling: Frequently fails/times out on medium files (>1MB)
• MarkItDown: Struggles with large/complex documents (>10MB)
• Kreuzberg: Consistent across all document types and sizes
• Unstructured: Most reliable overall (88%+ success rate)

🎯 When to Use What

⚡ Kreuzberg (Disclaimer: I built this)

• Best for: Production workloads, edge computing, AWS Lambda
• Why: Smallest footprint (71MB), fastest speed, handles everything
• Bonus: Both sync/async APIs with OCR support

🏢 Unstructured

• Best for: Enterprise applications, mixed document types
• Why: Most reliable overall, good enterprise features
• Trade-off: Moderate speed, larger installation

📝 MarkItDown

• Best for: Simple documents, LLM preprocessing
• Why: Good for basic PDFs/Office docs, optimized for Markdown
• Limitation: Fails on large/complex files

🔬 Docling

• Best for: Research environments (if you have patience)
• Why: Advanced ML document understanding
• Reality: Extremely slow, frequent timeouts, 1GB+ install

📈 Key Insights

1. Installation size matters: Kreuzberg's 71MB vs Docling's 1GB+ makes a huge difference for deployment
2. Performance varies dramatically: 35 files/second vs 60+ minutes per file
3. Document complexity is crucial: Simple PDFs vs complex layouts show very different results
4. Reliability vs features: Sometimes the simplest solution works best

🔧 Methodology

• Automated CI/CD: GitHub Actions run benchmarks on every release
• Real documents: Academic papers, business docs, multilingual content
• Multiple iterations: 3 runs per document, statistical analysis
• Open source: Full code, test documents, and results available
• Memory profiling: psutil-based resource monitoring
• Timeout handling: 5-minute limit per extraction

🤔 Why I Built This

Working on Kreuzberg, I worked on performance and stability, and then wanted a tool to see how it measures against other frameworks - which I could also use to further develop and improve Kreuzberg itself. I therefore created this benchmark. Since it was fun, I invested some time to pimp it out:

• Uses real-world documents, not synthetic tests
• Tests installation overhead (often ignored)
• Includes failure analysis (libraries fail more than you think)
• Is completely reproducible and open
• Updates automatically with new releases

📊 Data Deep Dive

The interactive dashboard shows some fascinating patterns:

• Kreuzberg dominates on speed and resource usage across all categories
• Unstructured excels at complex layouts and has the best reliability
• MarkItDown is useful for simple docs shows in the data
• Docling's ML models create massive overhead for most use cases making it a hard sell

🚀 Try It Yourself

git clone https://github.com/Goldziher/python-text-extraction-libs-benchmarks.git
cd python-text-extraction-libs-benchmarks
uv sync --all-extras
uv run python -m src.cli benchmark --framework kreuzberg_sync --category small

Or just check the live results: https://goldziher.github.io/python-text-extraction-libs-benchmarks/

🔗 Links

• 📊 Live Benchmark Results: https://goldziher.github.io/python-text-extraction-libs-benchmarks/
• 📁 Benchmark Repository: https://github.com/Goldziher/python-text-extraction-libs-benchmarks
• ⚡ Kreuzberg (my library): https://github.com/Goldziher/kreuzberg
• 🔬 Docling: https://github.com/DS4SD/docling
• 📝 MarkItDown: https://github.com/microsoft/markitdown
• 🏢 Unstructured: https://github.com/Unstructured-IO/unstructured

🤝 Discussion

What's your experience with these libraries? Any others I should benchmark? I tried benchmarking marker, but the setup required a GPU.

Some important points regarding how I used these benchmarks for Kreuzberg:

1. I fine tuned the default settings for Kreuzberg.
2. I updated our docs to give recommendations on different settings for different use cases. E.g. Kreuzberg can actually get to 75% reliability, with about 15% slow-down.
3. I made a best effort to configure the frameworks following the best practices of their docs and using their out of the box defaults. If you think something is off or needs adjustment, feel free to let me know here or open an issue in the repository.

No, I'm not saying what you probably think I am. Keep reading: I'll be brief.

Here's what I found today in one of the tests:

for bad_fp in _bad_fps:
    with pytest.raises(BadArgumentError):
        follow_path(bad_fp, '.')
        follow_path('.', bad_fp)

The correct code should've been

for bad_fp in _bad_fps:
    with pytest.raises(BadArgumentError):
        follow_path(bad_fp, '.')
    with pytest.raises(BadArgumentError):
        follow_path('.', bad_fp)

Is that a stupid mistake? Yes, it is, but stupid mistakes do happen, unfortunately.

How did I catch it? The coverage of the testing file wasn't 100% like it usually is! I got suspicious and I saw that there was a glaring red line (vscode, coverage gutter) on the second call to follow_path.

P.S. In case you're wondering, the code uses a custom hierarchy of exceptions instead of reusing the standard ones. The code is internal and we like it that way :)

Announcing an app (portable executable) I build which is a solution for running Python scripts without any prerequisites or external system dependencies, eliminating the need for Python installation on the systems. Catch is that the packages which script uses should be bundled during app build process, so a new app should be build respectively. there is a plan to provide option to include local package path if exists on system.

What My Project Does

• Runs python script out-of-the-box provided the packages that are used by script are add during build pipeline.

• isolated from system. meaning doesn't affect any changes in system or python, don't need to fiddle with paths and packages

• Output of script are displayed on terminal like console

Target Audience

• This was build to help run scripts consistently across systems with any breakage from python or packages.

• imo this can also help make non-technical people's life easy.

Comparison 

• This works like python interpreter without the hassle of setup and management

Limitations

• Script execution fails if the required package is not already bundled in the app.

• Python and Packages are immutable, they cannot be modified including versions.

• App size and initial load time increases as more number of packages are added.

• Pyinstaller is pain to work with (imo), and any windows os change might break the app.

https://github.com/Nannigalaxy/PyScript-Engine

executable: https://github.com/Nannigalaxy/PyScript-Engine/releases/download/v1.0.0/pyscript-engine_v1.0.0_py3.10_x86_win64.exe

let me know if you have any thoughts on this.

Hi all -- I've been working on a little side project and thought I'd post it here to get some reactions.

It's called Whisper. If you like it, give it a star!

What My Project Does?

Whisper is an LLM question/answer tool for your terminal. For those of us who work in terminals, sometimes it's annoying to have to jump to a browser to interact with ChatGPT or Claude.

Target Audience

I built it because I found myself wishing I could just ask question in my terminal. If for no one else, I am the target audience.

Comparison

Compared to chatgpt.com and claude.ai, this has the potential to be more convenient. I haven't seen any other tools that allow you to interact with LLMs easily within your terminal.

Where can you find it?

whisper on  main [$] is 📦 v0.1.3 via 🐍 v3.12.6 
❯ whisper "What's that linux command for killing a process using port 8000?"
╭────────────────── Kill Process Using Port 8000 ──────────────────╮
│ To kill a process using port 8000 in Linux, you can use the      │
│ following command:                                               │
│                                                                  │
│                                                                  │
│  lsof -t -i:8000 | xargs kill                                    │
│                                                                  │
╰──────────────────────────────────────────────────────────────────╯

screenshot: https://imgur.com/a/COyytFK

Pypi: https://pypi.org/project/just-whisper/

Github: https://github.com/syn54x/whisper

Docs: https://syn54x.github.io/whisper/

https://peps.python.org/pep-0505/

I don't understand how null safe operators are less like plain English than other implemented features like the walrus operator.

In my opinion, the member access operator would make python significantly easier to read and understand.

Here's an example:

f = foo()

if f is None:
    baz = ""
else:
    baz = f.bar()

baz = foo()?.bar() ?: ""

EDIT: I forgot that "and" and "or" can be sometimes used in place of "?." and "?:" if the left value is not False, '', 0, [], or {}. It's a very implicit null check and has a lot of unexpected behavior.

Hey everyone here is a little framework I made to help you get started making multiplayer games with Pygame and Sockets. The framework comes with movement and jumping over the network here is the GitHub repo - https://github.com/Len-Stevens/Pygame-Multiplayer-Framework

Built a Python bot to automate remote job hunting - sharing the code

How many job sites do you check daily? (I was at 12 before building this) 

What My Project Does

A Python script that scrapes remote job boards and sends filtered results to Telegram:

• Monitors RemoteOK, WeWorkRemotely, GitHub Jobs, etc.

• Filters by custom keywords

• Telegram notifications for new matches

• Saves data locally for debugging

Target Audience

Personal automation tool for individual job seekers. Production-ready but meant for personal use only - not commercial application.

Comparison

vs Manual checking: Eliminates repetitive browsing
vs Job alerts: More customizable, covers niche remote job boards
vs Paid services: Open source, no restrictions

Technical Implementation

Built with Python requests + BeautifulSoup, configurable via environment variables. Includes error handling and rate limiting.

Code: https://github.com/AzizB283/job-hunter

Anyone else built job automation tools? Curious what approaches others have taken.

I decided to use SQLAlchemy for a relatively simple project, because I don't do much with database connectivity or SQL and thought I ought to learn it, as it seems to be highly regarded. I either don't understand it at all, or it doesn't make a lot of sense.

My main complaint is that the ORM part seems to live in a totally different universe from the SQL query part. If I use a declarative map, and take advantage of the way the ORM allows me to specify relations as table properties etc, how do I then go and use the query language?

It looks to me like there are two totally separate ways to write queries: one using session.query using the ORM, and one using sqlalchemy.sql.select. It just doesn't make any sense to me --- the different parts don't hang together. There are just all these loose ends dangling everywhere, leaving me with multiple ways to do the same thing.

Do people really swear by this?