Comments for Andrew M. Whalen

Comment on Finding Frequency Counts With SQL by boblangdon

boblangdon — Thu, 19 Jan 2012 23:05:45 +0000

Ditto!
As usual, the solution appears simple, once I find it.

Perfect to grab most popular search terms from our cookies-data table.

Comment on Archive My Tweets by Alexandre Giesbrecht

Alexandre Giesbrecht — Thu, 05 Jan 2012 17:17:28 +0000

Correction: it wasn’t the script’s fault that some dates were imported incorrectly: it was Twitter’s fault (http://twitter.com/#!/agiesbrecht/status/6459629815).

Also, the tweets were imported in the first step, when I got the “maximum execution time” error.

Second try gathered 1,794 tweets, but it went back the same length of time of the first try (10/1/2009). It’s just that I had deleted many more tweets in the mean time. Is this because of the Twitter API or something that’s coded?

I’m not looking for a fix, just reporting it so someone might have their doubt answered here. Sorry that I have flooded the comments!

Comment on Archive My Tweets by Alexandre Giesbrecht

Alexandre Giesbrecht — Thu, 05 Jan 2012 16:12:17 +0000

Out of the blue, I went to the page created, and the tweets were there (and, of course, they are in the DB as well). So that must be why it was looking for something greater (more recent) than my latest tweet.

It imported 1,919 of my 3,800+ tweets, and it got a few dates wrong (Dec. 1982, Oct. 1988, Oct. 2014, Mar. 2021, Mar. 2027 and Apr. 2028). Other than that, it was perfect. I’ll give it a try again, renaming the current table and generating a new one.

I can’t say thank you enough, Andrew!

Comment on Archive My Tweets by Alexandre Giesbrecht

Alexandre Giesbrecht — Thu, 05 Jan 2012 12:20:58 +0000

Andrew, I can’t say enough good things about your help, so I don’t know how to say this, but… it didn’t work again.

At first I got the same “maximum execution time” error message, but I refreshed the page and it seemed to start. Yet…:

Retrieving 200 tweets per page.
Getting tweets with an id greater than 154876345120342016.
No tweets on page 1.
0 new tweets over 1 query.
API: (328/350 remaining) API count resets at 11:05am America/Sao_Paulo.

154876345120342016 is the ID of my latest tweet. Maybe that’s it?

(On a related-but-unrelated note, I started deleting many tweets in order to try to get to 3,000. I got from 4,075 to a little less than 4,000, but, boy, how it’s hard to delete an old tweet in the current format! As soon as you delete one, you get all the way back to almost top.)

Thanks again, Andrew!

Comment on Archive My Tweets by Andrew

Andrew — Thu, 05 Jan 2012 00:14:05 +0000

Alexandre, I think you’re almost there. I added a slightly new version (0.4). It will add the tweets to your database after fetching every 200 results rather than saving the database insertion until the very end.

Comment on Archive My Tweets by Alexandre Giesbrecht

Alexandre Giesbrecht — Wed, 04 Jan 2012 23:07:11 +0000

Thanks (again), Andrew. But… it hasn’t worked again.

The first time I run the page, I get this message: “PHP Fatal error: Maximum execution time of 90 seconds exceeded in E:\home\pittsburgh\Web\tweets\classes\twitter.php on line 473″. It does create the table on my database, though (I had dropped it).

If I refresh the page, the message changes to this or some variation of it:

Retrieving 200 tweets per page.
Got 200 results on page 1.
Got 200 results on page 2.
Got 200 results on page 3.
Got 200 results on page 4.
Got 200 results on page 5.
Got 200 results on page 6.
Got 200 results on page 7.
Got 200 results on page 8.
Got 200 results on page 9.
Got 200 results on page 10.
Got 200 results on page 11.
Exception: Invalid response.
Got 200 results on page 12.
Exception: Invalid response.
Got 199 results on page 13. (Some tweets may have been filtered out.)
Got 200 results on page 14.
Exception: Invalid response.
Got 200 results on page 15.
Exception: Invalid response.
Got 200 results on page 16.
No tweets on page 17.
3199 new tweets over 17 queries.
API: (308/350 remaining) API count resets at 9:51pm America/Sao_Paulo.
ERROR INSERTING INTO DATABASE: MySQL server has gone away

Table remains empty after it: http://screencast.com/t/OvwxDLkOt

I do have more than 3,200 tweets (4,074). Maybe that’s an issue?

Again, your effort has been very much appreciated!

Comment on Archive My Tweets by Regs

Regs — Wed, 04 Jan 2012 22:14:11 +0000

Andrew, you’re a champion. It’s all working now.

Thanks for the update.

Comment on Archive My Tweets by Andrew

Andrew — Wed, 04 Jan 2012 05:02:52 +0000

Also, retweets are now included. Unfortunately, this will only apply to new tweets as they’re added.

If you have less than 3200 tweets (the maximum that can be retrieved from the Twitter API), you can empty your database table an re-run your cron.php file. All your tweets will then be reimported.

Comment on Archive My Tweets by Andrew

Andrew — Wed, 04 Jan 2012 04:57:53 +0000

I made some more additions, updates, and fixes. Download version 0.3 here: http://code.google.com/p/archive-my-tweets/downloads/list. As always just replace all the files with the new ones except for your custom config.php.

I fixed an API parameter bug which will hopefully fix the problem with everyone getting zero rows inserted into their databases. I added paging to the ‘all tweets’ view, cleaned up some error messages, removed some deprecated functions, and raised the twitter api exception limit to 25 from 5.

Comment on Archive My Tweets by Alexandre Giesbrecht

Alexandre Giesbrecht — Tue, 03 Jan 2012 22:26:17 +0000

Thanks for the update, Andrew. It didn’t work for me, though. The message changed, and I got the dreaded ERROR INSERTING INTO DATABASE message:

Retrieving 200 tweets per page.
No tweets on page 1.
0 new tweets over 1 query.
API: (347/350 remaining) API count resets at 9:22pm America/Sao_Paulo.
ERROR INSERTING INTO DATABASE:

Also, with the previous version it took always something like a minute for the messages to appear; now it’s almost instantaneous.

Any clue?