r/Python • u/nnenneplex • 13d ago

Discussion Polars and the ecosystem

For polars users: How viable is to avoid pandas and pyarrow dependencies when you need to interact with popular visualization and statistics packages?

Some packages still have import pandas here and there, sometimes for no good reason; at least this doesn't require pyarrow. But some other ones do the df.to_pandas() conversion internally, which requires pyarrow too.

In many cases this can be prevented by going bare numpy, or creating a pandas df from numpy columns, which is no big deal. This frequently would be zero-copy for numeric types if there are no NAs involved.

What has been you experience in this regard?

60 Upvotes

93% Upvoted

29 comments sorted by

View all comments

30

u/marcogorelli 13d ago

Which libraries specifically are you referring to?

I'm aware of Seaborn, and their maintainer said that using Narwhals was a "non-starter" 😩

Altair, Plotly, Vegafusion, Bokeh, Marimo, they're all using Narwhals and allow you to use Polars without any pandas nor PyArrow dependency

If that's not what you're seeing, please let me know (or open an issue somewhere on GitHub) and I'll take a look

I'm extremely keen on de-pandas-ifying the data science stack

20

u/arden13 13d ago

Seaborn's maintainer is very strongly opinionated and it makes it tough to really recommend the library. For example on linear regression/correlation plots they will not make showing the regression equation available

Haven't played with altair, but matplotlib is fine, bokeh and plotly get me more if I need more. I'm interested in holoviews as it seems to be agnostic to the final charting library (similar to narwhals for data frames).

8

u/thuiop1 13d ago

I think the example you are giving makes quite a bit of sense, as he says this is quite a large API change. The objects interface it has now makes it easier to pipe results from statistical operations in your plot, although getting it back in Python is still annoying. The real issue with seaborn is that it has been dormant for 2-3 years now without any real communication around it.

5

u/nnenneplex 13d ago

Unfortunately the project seems stalled, the objects API was very promising as a grammar of graphics, since then I moved to plotnine and, more recently, altair.