Usually, a crushed soda can is good enough to prevent the kickstand to sync in the pavement. You can usually find a can in a random parking lot. That, or find a strip of concrete. That's why sometimes motorcycle park on the sidewalk in front of big box stores on a hot day.
My first reaction is that it depends on who you read.
The linked article talks about Wilhelm von Humboldt's philosophy of education. While I haven't read much into 19th-century German literature, the article seems to suggest that a national education system is foundational in nation building and, possibly on-brand romanticism, that the final goal of education is to produce "independent, critical thinkers".
The same ideals have driven the initial push for public schooling in the United States (which happened at the same time at least in the big East Coast cities). However, with the expansion west, schooling became more of an assimilation instrument, where the preparation of "informed citizens" became more of the goal. This led to public school clashes with established religious schools (mostly Catholic in Chicago and in California), which then resulted in a full separation between public and religious school funding.
The goal of education seemed to have changed with the beginning of the 20th century and the push for universal high school. Powell, Farrar, & Cohen argue in "Shopping Mall High School"[^1] that universal secondary education forced schools to become more “consumer-oriented" by offering classes and activities (i.e., sports) that would keep students in school until 18, while compromising with their original ideals to prepare citizens or critical thinkers.
This fits a need I had when working with DuckDB: running statistical analyses directly within the database without having to spin up external tools (like R/Stata/Python).
I really appreciate the API for calling stats functions and retrieving results. This seamless integration was exactly what I was missing.
Regarding my concerns, the project currently gets you through about a first semester's worth of grad-level statistics or quantitative methods. It's sufficient for exploratory descriptive statistics, paired tests, and basic linear regressions. However, this also means it isn't close to the "production-level" statistics required for rigorous research work. At a minimum, it needs to support heteroskedastic-robust standard errors (Huber-White and clustered at a minimum; jackknife and bootstrapped as a non-parametric bonus), multilevel linear models, and generalized linear models (GLMs). I notice there is an open issue for GLM support, though it sounds like that will require a full rewrite of the inference backend. Including marginal estimates following a regression would also be highly useful, especially if GLMs are implemented.
Taking this library from an MVP to a production-ready replacement for SAS, R, or Stata will require significant effort. I am unsure about the market fit for a tool like this; organizations paying for SAS or Stata are unlikely to abandon them for an upstart project, and R has a deeply entrenched ecosystem that is impossible to replace. The situation feels similar to the dynamic between Octave and Matlab, or SageMath and Mathematica. It risks becoming a free alternative used primarily by those who cannot afford the paid products.
As others in the thread have pointed out, this extension includes functionality that is already handled well by other community extensions (like ggsql, stochastic, and read_stat). Because the ultimate goal seems to be providing a SAS-compatible frontend built on DuckDB, which is a huge and exciting undertaking. I wonder if the statistical backend might progress faster by focusing only on tests and regressions. Since it serves as the foundation for the other frontend services, zeroing in on the core stats and relying on the broader DuckDB ecosystem for the rest might make this massive scope a bit more manageable.
Thank you, that's quite a lot of great quality feedback!
Agreed, it's not production-grade yet, and robust standard errors are the priority. HC and cluster-robust SEs are the biggest credibility gap, which is why I just added an (Eigen-based) linalg kernel for the next release: it's the groundwork so the regression fitter can ship HC0–HC3 and clustered SEs (and bootstrapped, via the existing bootstrap aggregate). Margins and GLMs (IRLS on the same kernel, so not really a backend rewrite) are the next layers.
About the scope, agreed, and already narrowing. I'll credit posit ggsql as the real grammar-of-graphics tool; mine stays a minimal built-in. The plan is to go deep on the statistical core and lean on the DuckDB ecosystem for the rest.
For the market fit I'd frame it a bit differently. Not trying to pry anyone off Stata/R for rigorous work, I know that those aren't going anywhere soon. The niche is stats where the data already lives: people in SQL/DuckDB who currently round-trip to R/Python for a regression. R can run in the browser now (webR), but not in the same engine as the data. The stats-duck runs inside duckdb-wasm, so there's no separate runtime and no marshalling. And DuckDB is already far faster than R for the ETL/wrangling around the stats.
The plan is to figure out from real usage. For now, I'll focus on the core.
The figure-8 central design makes this roundabout not a real roundabout. The challenge is that the entry and exit points are at the tightest part of the curve, making ingress and egress kind of weird. It looks like they did it because of space constraints on the bridge. What I find weird is the tightness of the on/off ramp, which share a very small part of the central lanes. In a traditional circle, they are spaced more apart, making exiting and entering smoother.
If it were a regular 2-lane traffic circle, it wouldn't be too difficult to navigate.
Personally, I find multiple-lane traffic circles (4+) to be more difficult to navigate, mostly when some people in the center lanes pull a fast right turn across all lanes of traffic. Otherwise, much better than traffic lights at keeping traffic moving or first-come, first-served stop intersections when people don't really stop or sit there waving you through.
This seems to be a problem with the Stata code. Disappointing but not really surprising. Stata isn't an easy language to debug and problems like this happen a lot. I wonder how much coding was done by the original author versus being coded by a research assistant.
I think that some of the biggest issues with omitted variables could have been address by using the "new" Stata interaction syntax ("new" in the sense that has been available in Stata since 10 releases ago) instead of rolling their own products and forgetting to include the main effect.
Regarding public comments, I don't believe a good politician will make a snap decision at the dais following public comments. Most of them will have received the meeting agenda in advance and formed an opinion about how they are going to vote and the questions they are going to ask. If this is the case, public comment is just a waste of time for them, as they won't really get swayed by it. At most, they will mention a point that a public commenter made to support something that they were already going to do.
Emailing them privately in advance of the meeting will give them the opportunity to think about your input and, in some cases, reply and engage with you about the policy. It might not change their mind, but it will definitely help them see others' perspectives on their upcoming decision.
In the best case, you're betting on some subset of whatever board showing up not having counted the votes they need for their preferred outcome. If something's on the agenda, it's because the board wanted it on the agenda!
The ironic part about these hallucinations is that a research paper includes a literature review because the goal of the research is to be in dialogue with prior work, to show a gap in the existing literature, and to further the knowledge that this prior work has built.
By using an LLM to fabricate citations, authors are moving away from this noble pursuit of knowledge built on the "shoulders of giants" and show that behind the curtain output volume is what really matters in modern US research communities.
That's going to be the philosophical question of our times: do LLMs generate slop out of nowhere or does it simply amplify the slop machinery that was already there?
Neither Python or R does a good job at all of these.
The original article seems to focus on challenges in using Python for data preparation/processing, mostly pointing out challenges with Pandas and "raw" Python code for data processing.
This could be solved by switching to something like duckdb and SQL to process data.
As far as data analysis, both Python and R have their own niches, depending on field. Similarly, there are other specialized languages (e.g., SAS, Matlab) that are still used for domain-specific applications.
I personally find result preparation somewhat difficult in both Python and R. Stargazer is ok for exporting regression tables but it's not really that great. Graphing is probably better in R within the ggplot universe (I'm aware of the python port).
I set up something similar at work. But it was before the DuckLake format was available, so it just uses manually generated Parquet files saved to a bucket and a light DuckDB catalog that uses views to expose the parquet files. This lets us update the Parquet files using our ETL process and just refresh the catalog when there is a schema change.
We didn't find the frozen DuckLake setup useful for our use case. Mostly because the frozen catalog kind of doesn't make sense with the DuckLake philosophy and the cost-benefit wasn't there over a regular duckdb catalog. It also made making updates cumbersome because you need to pull the DuckLake catalog, commit the changes, and re-upload the catalog (instead of just directly updating the Parquet files). I get that we are missing the time travel part of the DuckLake, but that's not critical for us and if it becomes important, we would just roll out a PostgreSQL database to manage the catalog.
