Comments on: Using a simple Python script for End-to-End Data Transformation and ETL (Part 1) http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/ Site Fabricator, Data Welder, Heavy Metal Systems Integrator | I.Make.Shit.Work. Wed, 14 Sep 2011 11:14:02 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Andy B http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-754 Andy B Mon, 27 Jun 2011 10:34:54 +0000 http://ryrobes.com/?p=91#comment-754 I am having a similar problem to Joel. I have implemented similar code, but am receiving the error "ValueError: expected simple type, a tuple or a list." Looking at the local variables from my debug session, I am providing a list of tuples, so not sure what I am doing wrong. Any help appreciated I am having a similar problem to Joel. I have implemented similar code, but am receiving the error “ValueError: expected simple type, a tuple or a list.” Looking at the local variables from my debug session, I am providing a list of tuples, so not sure what I am doing wrong.

Any help appreciated

]]> By: Ry http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-747 Ry Wed, 01 Jun 2011 22:39:37 +0000 http://ryrobes.com/?p=91#comment-747 Hey Joel, sorry for the delay. Still having issues? Hey Joel, sorry for the delay. Still having issues?

]]> By: Joel http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-743 Joel Thu, 05 May 2011 00:00:08 +0000 http://ryrobes.com/?p=91#comment-743 Say I am quite intrigued by this sample - I am new to Python and am a GIS analyst. I have been tasked with running a GIS process that applies spatial attributes - then I have to pull the results from SQLEXPRESS database and write to a waiting table on another SQL Server. I am at my wits end since I have implemented your code and I know I am retrieveing but I keep getting an error stating that it is looking for a a tuple or a dictionary. Any help would be greatly appreciated. Thanks, Joel Hanson Say I am quite intrigued by this sample – I am new to Python and am a GIS analyst. I have been tasked with running a GIS process that applies spatial attributes – then I have to pull the results from SQLEXPRESS database and write to a waiting table on another SQL Server. I am at my wits end since I have implemented your code and I know I am retrieveing but I keep getting an error stating that it is looking for a a tuple or a dictionary. Any help would be greatly appreciated.

Thanks,

Joel Hanson

]]> By: Pranav http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-729 Pranav Thu, 04 Nov 2010 08:55:37 +0000 http://ryrobes.com/?p=91#comment-729 Nicely written post. Have subscribed using Google Reader for RSS. Nicely written post. Have subscribed using Google Reader for RSS.

]]> By: How to run your Python scripts as a Windows Service | Ryan Robitaille http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-311 How to run your Python scripts as a Windows Service | Ryan Robitaille Tue, 17 Aug 2010 23:53:34 +0000 http://ryrobes.com/?p=91#comment-311 [...] any Python duct-taper integrate-anything junkie like me has a need to schedule their things (in production) every once in awhile. Usually this is [...] [...] any Python duct-taper integrate-anything junkie like me has a need to schedule their things (in production) every once in awhile. Usually this is [...]

]]> By: Business Intelligence, Tools, Dirty Caveman Sex, Open-Source - Part 1 http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-212 Business Intelligence, Tools, Dirty Caveman Sex, Open-Source - Part 1 Tue, 10 Aug 2010 05:15:48 +0000 http://ryrobes.com/?p=91#comment-212 [...] done it several times before using some custom (and clever – if I do say so myself) PHP, Python, and a variety of databases – but that was after I already had an intimate familiarity with [...] [...] done it several times before using some custom (and clever – if I do say so myself) PHP, Python, and a variety of databases – but that was after I already had an intimate familiarity with [...]

]]> By: Ry http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-130 Ry Tue, 04 May 2010 03:03:17 +0000 http://ryrobes.com/?p=91#comment-130 Hey Jorge, thanks for the comments! I guess I've just been spoiled over the past few years since I'd create the datasource AND do the report afterward. :) I've also def warmed up to the traditional DW methods for a few larger projects recently. I guess sometimes we just get so used to doing everything "one way", new (well, old) concepts seem like more effort than payoff. Also, I LOVE how I can deploy some well-thought-out Analysis Service Cubes to my end-users, then they can manipulate them as pivot tables in Excel on their own. It saves me the trouble of writing several specific reports off the same data... (you know, the "classic" way) :) Cheers! Hey Jorge, thanks for the comments!

I guess I’ve just been spoiled over the past few years since I’d create the datasource AND do the report afterward. :)

I’ve also def warmed up to the traditional DW methods for a few larger projects recently. I guess sometimes we just get so used to doing everything “one way”, new (well, old) concepts seem like more effort than payoff.

Also, I LOVE how I can deploy some well-thought-out Analysis Service Cubes to my end-users, then they can manipulate them as pivot tables in Excel on their own. It saves me the trouble of writing several specific reports off the same data… (you know, the “classic” way)

:)

Cheers!

]]> By: Jorge http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-129 Jorge Fri, 30 Apr 2010 18:36:10 +0000 http://ryrobes.com/?p=91#comment-129 I know this is an old post. I come from the end of the spectrum of being Report Developer - I found that creating reports and writing SQL off of a set of tables following Kimbals methods - star schema, as a positive experience! Whats great about it is that it is a universal structure. When I have encountered a star schema - I instantly know what tables I need to join and have hit the ground running - whereas compared to a table structure someone has invented - I always have to go to the inventor and ask how these tables are joined, whats the granularity and what not. Also I found working with a cube as being positve thing to. Writing a report thats crunching and aggregating year over year sales data coming from 2,500 plus stores is extremely fast (milleseconds) compared to the same SQL. I know this is an old post.

I come from the end of the spectrum of being Report Developer – I found that creating reports and writing SQL off of a set of tables following Kimbals methods – star schema, as a positive experience!

Whats great about it is that it is a universal structure. When I have encountered a star schema – I instantly know what tables I need to join and have hit the ground running – whereas compared to a table structure someone has invented – I always have to go to the inventor and ask how these tables are joined, whats the granularity and what not.

Also I found working with a cube as being positve thing to. Writing a report thats crunching and aggregating year over year sales data coming from 2,500 plus stores is extremely fast (milleseconds) compared to the same SQL.

]]> By: Tim Day http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-123 Tim Day Fri, 26 Feb 2010 06:32:45 +0000 http://ryrobes.com/?p=91#comment-123 Thanks a lot for the post. I used pyodbc for connecting to an msaccess db and dragging things into oracle. I really enjoyed not having to touch access. I wonder if there is a way to get the table and column names so that we can also build the required target tables automatically? Thanks a lot for the post. I used pyodbc for connecting to an msaccess db and dragging things into oracle. I really enjoyed not having to touch access. I wonder if there is a way to get the table and column names so that we can also build the required target tables automatically?

]]> By: Using XLWT and Python to export an Oracle dataset to Excel (Python Simple ETL Part 2) | Ryan Robitaille http://ryrobes.com/featured-articles/using-a-simple-python-script-for-end-to-end-data-transformation-and-etl-part-1/comment-page-1/#comment-81 Using XLWT and Python to export an Oracle dataset to Excel (Python Simple ETL Part 2) | Ryan Robitaille Tue, 08 Dec 2009 04:31:26 +0000 http://ryrobes.com/?p=91#comment-81 [...] are part of the standard Python distriution (I’m using 2.6 in this example), cx_Oracle I already discussed HERE, and xlwt can be found HERE (http://pypi.python.org/pypi/xlwt) grab the packages for your platform [...] [...] are part of the standard Python distriution (I’m using 2.6 in this example), cx_Oracle I already discussed HERE, and xlwt can be found HERE (http://pypi.python.org/pypi/xlwt) grab the packages for your platform [...]

]]>