I guess I’ve just been spoiled over the past few years since I’d create the datasource AND do the report afterward. :)

I’ve also def warmed up to the traditional DW methods for a few larger projects recently. I guess sometimes we just get so used to doing everything “one way”, new (well, old) concepts seem like more effort than payoff.

Also, I LOVE how I can deploy some well-thought-out Analysis Service Cubes to my end-users, then they can manipulate them as pivot tables in Excel on their own. It saves me the trouble of writing several specific reports off the same data… (you know, the “classic” way)

:)

Cheers!

I come from the end of the spectrum of being Report Developer – I found that creating reports and writing SQL off of a set of tables following Kimbals methods – star schema, as a positive experience!

Whats great about it is that it is a universal structure. When I have encountered a star schema – I instantly know what tables I need to join and have hit the ground running – whereas compared to a table structure someone has invented – I always have to go to the inventor and ask how these tables are joined, whats the granularity and what not.

Also I found working with a cube as being positve thing to. Writing a report thats crunching and aggregating year over year sales data coming from 2,500 plus stores is extremely fast (milleseconds) compared to the same SQL.

Yeah, I think you’ve got the gist of my problem.

So, I had been thinking about using an ETL tool (such as Talend) – but some Python people had discouraged me from that, saying that ETL tools were for people who didn’t know how to program, they came with their own product-specific learning curve, and they weren’t necessarily more rapid than just writing your own scripts at the end of the day.

So what do you think of using an ETL tool vs. writing your own scripts (like you demo in this article).

Currently I’m leaning toward building what I’m calling a “pseudo-datawarehouse”, forgetting about Inman and Kimball, and not looking for a cross-vendor join solution.

I guess it depends on what you’re trying to do exactly. If your data is so spread out all over the place, it probably should all run from a main location (all in the same DB) – and then be pushed back out the the old locations on a scheduled basis (if they even need to exist anymore).

I’ve gone down the rabbit hole on building data warehouses (and datamarts) and data cubes to report on and all that fancy “best practice” stuff… but I honestly think that for a lot of implementations its total overkill. That’s just my humble opinion.

You could look at a more “Access friendly” ETL tool like SQL Server SSIS, but its costs a pretty penny. If you can find a desktop version of SQL Server 2000 somewhere – download and use that, back then they included the whole ETL tool in that version and i’ve been using it like a “data swiss army knife” ever since.

Let me know if I understand the problem right. :)

I’ve been getting all worked up about data warehousing and ETL for my project, where I’ve gotta do stuff like build an interface for creating new order and order_item entities that relate to members (MS Access), employees (MySQL) and products (flat file).

People have been telling me I need to read up on Inmon and Kimball and understand stuff like star schemas, data marts, EAV tables and so on.

Do you think all I need to do is write a bunch of python ETL scripts with event triggers (like “create new order” updates all the related tables in the warehouse)?

And why not bypass the ETL layer entirely and just perform cross-vendor joins on the application level?

Thanks in advance for any tips.

cursor.rowcount would be easier than the fetchone and var[0] printing – I might have to change that, since I’m trying to make it as SIMPLE as possible. I guess my brain is hardwired to do a lot of things the “hard way”.

Thanks again, Speno!