Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
View | Details | Raw Unified | Return to bug 189000 | Differences between
and this patch

Collapse All | Expand All

(-)flickrfs-1.3.9.orig/COPYING (+340 lines)
Line 0 Link Here
1
		    GNU GENERAL PUBLIC LICENSE
2
		       Version 2, June 1991
3
4
 Copyright (C) 1989, 1991 Free Software Foundation, Inc.
5
                       59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
6
 Everyone is permitted to copy and distribute verbatim copies
7
 of this license document, but changing it is not allowed.
8
9
			    Preamble
10
11
  The licenses for most software are designed to take away your
12
freedom to share and change it.  By contrast, the GNU General Public
13
License is intended to guarantee your freedom to share and change free
14
software--to make sure the software is free for all its users.  This
15
General Public License applies to most of the Free Software
16
Foundation's software and to any other program whose authors commit to
17
using it.  (Some other Free Software Foundation software is covered by
18
the GNU Library General Public License instead.)  You can apply it to
19
your programs, too.
20
21
  When we speak of free software, we are referring to freedom, not
22
price.  Our General Public Licenses are designed to make sure that you
23
have the freedom to distribute copies of free software (and charge for
24
this service if you wish), that you receive source code or can get it
25
if you want it, that you can change the software or use pieces of it
26
in new free programs; and that you know you can do these things.
27
28
  To protect your rights, we need to make restrictions that forbid
29
anyone to deny you these rights or to ask you to surrender the rights.
30
These restrictions translate to certain responsibilities for you if you
31
distribute copies of the software, or if you modify it.
32
33
  For example, if you distribute copies of such a program, whether
34
gratis or for a fee, you must give the recipients all the rights that
35
you have.  You must make sure that they, too, receive or can get the
36
source code.  And you must show them these terms so they know their
37
rights.
38
39
  We protect your rights with two steps: (1) copyright the software, and
40
(2) offer you this license which gives you legal permission to copy,
41
distribute and/or modify the software.
42
43
  Also, for each author's protection and ours, we want to make certain
44
that everyone understands that there is no warranty for this free
45
software.  If the software is modified by someone else and passed on, we
46
want its recipients to know that what they have is not the original, so
47
that any problems introduced by others will not reflect on the original
48
authors' reputations.
49
50
  Finally, any free program is threatened constantly by software
51
patents.  We wish to avoid the danger that redistributors of a free
52
program will individually obtain patent licenses, in effect making the
53
program proprietary.  To prevent this, we have made it clear that any
54
patent must be licensed for everyone's free use or not licensed at all.
55
56
  The precise terms and conditions for copying, distribution and
57
modification follow.
58
59
		    GNU GENERAL PUBLIC LICENSE
60
   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
61
62
  0. This License applies to any program or other work which contains
63
a notice placed by the copyright holder saying it may be distributed
64
under the terms of this General Public License.  The "Program", below,
65
refers to any such program or work, and a "work based on the Program"
66
means either the Program or any derivative work under copyright law:
67
that is to say, a work containing the Program or a portion of it,
68
either verbatim or with modifications and/or translated into another
69
language.  (Hereinafter, translation is included without limitation in
70
the term "modification".)  Each licensee is addressed as "you".
71
72
Activities other than copying, distribution and modification are not
73
covered by this License; they are outside its scope.  The act of
74
running the Program is not restricted, and the output from the Program
75
is covered only if its contents constitute a work based on the
76
Program (independent of having been made by running the Program).
77
Whether that is true depends on what the Program does.
78
79
  1. You may copy and distribute verbatim copies of the Program's
80
source code as you receive it, in any medium, provided that you
81
conspicuously and appropriately publish on each copy an appropriate
82
copyright notice and disclaimer of warranty; keep intact all the
83
notices that refer to this License and to the absence of any warranty;
84
and give any other recipients of the Program a copy of this License
85
along with the Program.
86
87
You may charge a fee for the physical act of transferring a copy, and
88
you may at your option offer warranty protection in exchange for a fee.
89
90
  2. You may modify your copy or copies of the Program or any portion
91
of it, thus forming a work based on the Program, and copy and
92
distribute such modifications or work under the terms of Section 1
93
above, provided that you also meet all of these conditions:
94
95
    a) You must cause the modified files to carry prominent notices
96
    stating that you changed the files and the date of any change.
97
98
    b) You must cause any work that you distribute or publish, that in
99
    whole or in part contains or is derived from the Program or any
100
    part thereof, to be licensed as a whole at no charge to all third
101
    parties under the terms of this License.
102
103
    c) If the modified program normally reads commands interactively
104
    when run, you must cause it, when started running for such
105
    interactive use in the most ordinary way, to print or display an
106
    announcement including an appropriate copyright notice and a
107
    notice that there is no warranty (or else, saying that you provide
108
    a warranty) and that users may redistribute the program under
109
    these conditions, and telling the user how to view a copy of this
110
    License.  (Exception: if the Program itself is interactive but
111
    does not normally print such an announcement, your work based on
112
    the Program is not required to print an announcement.)
113
114
These requirements apply to the modified work as a whole.  If
115
identifiable sections of that work are not derived from the Program,
116
and can be reasonably considered independent and separate works in
117
themselves, then this License, and its terms, do not apply to those
118
sections when you distribute them as separate works.  But when you
119
distribute the same sections as part of a whole which is a work based
120
on the Program, the distribution of the whole must be on the terms of
121
this License, whose permissions for other licensees extend to the
122
entire whole, and thus to each and every part regardless of who wrote it.
123
124
Thus, it is not the intent of this section to claim rights or contest
125
your rights to work written entirely by you; rather, the intent is to
126
exercise the right to control the distribution of derivative or
127
collective works based on the Program.
128
129
In addition, mere aggregation of another work not based on the Program
130
with the Program (or with a work based on the Program) on a volume of
131
a storage or distribution medium does not bring the other work under
132
the scope of this License.
133
134
  3. You may copy and distribute the Program (or a work based on it,
135
under Section 2) in object code or executable form under the terms of
136
Sections 1 and 2 above provided that you also do one of the following:
137
138
    a) Accompany it with the complete corresponding machine-readable
139
    source code, which must be distributed under the terms of Sections
140
    1 and 2 above on a medium customarily used for software interchange; or,
141
142
    b) Accompany it with a written offer, valid for at least three
143
    years, to give any third party, for a charge no more than your
144
    cost of physically performing source distribution, a complete
145
    machine-readable copy of the corresponding source code, to be
146
    distributed under the terms of Sections 1 and 2 above on a medium
147
    customarily used for software interchange; or,
148
149
    c) Accompany it with the information you received as to the offer
150
    to distribute corresponding source code.  (This alternative is
151
    allowed only for noncommercial distribution and only if you
152
    received the program in object code or executable form with such
153
    an offer, in accord with Subsection b above.)
154
155
The source code for a work means the preferred form of the work for
156
making modifications to it.  For an executable work, complete source
157
code means all the source code for all modules it contains, plus any
158
associated interface definition files, plus the scripts used to
159
control compilation and installation of the executable.  However, as a
160
special exception, the source code distributed need not include
161
anything that is normally distributed (in either source or binary
162
form) with the major components (compiler, kernel, and so on) of the
163
operating system on which the executable runs, unless that component
164
itself accompanies the executable.
165
166
If distribution of executable or object code is made by offering
167
access to copy from a designated place, then offering equivalent
168
access to copy the source code from the same place counts as
169
distribution of the source code, even though third parties are not
170
compelled to copy the source along with the object code.
171
172
  4. You may not copy, modify, sublicense, or distribute the Program
173
except as expressly provided under this License.  Any attempt
174
otherwise to copy, modify, sublicense or distribute the Program is
175
void, and will automatically terminate your rights under this License.
176
However, parties who have received copies, or rights, from you under
177
this License will not have their licenses terminated so long as such
178
parties remain in full compliance.
179
180
  5. You are not required to accept this License, since you have not
181
signed it.  However, nothing else grants you permission to modify or
182
distribute the Program or its derivative works.  These actions are
183
prohibited by law if you do not accept this License.  Therefore, by
184
modifying or distributing the Program (or any work based on the
185
Program), you indicate your acceptance of this License to do so, and
186
all its terms and conditions for copying, distributing or modifying
187
the Program or works based on it.
188
189
  6. Each time you redistribute the Program (or any work based on the
190
Program), the recipient automatically receives a license from the
191
original licensor to copy, distribute or modify the Program subject to
192
these terms and conditions.  You may not impose any further
193
restrictions on the recipients' exercise of the rights granted herein.
194
You are not responsible for enforcing compliance by third parties to
195
this License.
196
197
  7. If, as a consequence of a court judgment or allegation of patent
198
infringement or for any other reason (not limited to patent issues),
199
conditions are imposed on you (whether by court order, agreement or
200
otherwise) that contradict the conditions of this License, they do not
201
excuse you from the conditions of this License.  If you cannot
202
distribute so as to satisfy simultaneously your obligations under this
203
License and any other pertinent obligations, then as a consequence you
204
may not distribute the Program at all.  For example, if a patent
205
license would not permit royalty-free redistribution of the Program by
206
all those who receive copies directly or indirectly through you, then
207
the only way you could satisfy both it and this License would be to
208
refrain entirely from distribution of the Program.
209
210
If any portion of this section is held invalid or unenforceable under
211
any particular circumstance, the balance of the section is intended to
212
apply and the section as a whole is intended to apply in other
213
circumstances.
214
215
It is not the purpose of this section to induce you to infringe any
216
patents or other property right claims or to contest validity of any
217
such claims; this section has the sole purpose of protecting the
218
integrity of the free software distribution system, which is
219
implemented by public license practices.  Many people have made
220
generous contributions to the wide range of software distributed
221
through that system in reliance on consistent application of that
222
system; it is up to the author/donor to decide if he or she is willing
223
to distribute software through any other system and a licensee cannot
224
impose that choice.
225
226
This section is intended to make thoroughly clear what is believed to
227
be a consequence of the rest of this License.
228
229
  8. If the distribution and/or use of the Program is restricted in
230
certain countries either by patents or by copyrighted interfaces, the
231
original copyright holder who places the Program under this License
232
may add an explicit geographical distribution limitation excluding
233
those countries, so that distribution is permitted only in or among
234
countries not thus excluded.  In such case, this License incorporates
235
the limitation as if written in the body of this License.
236
237
  9. The Free Software Foundation may publish revised and/or new versions
238
of the General Public License from time to time.  Such new versions will
239
be similar in spirit to the present version, but may differ in detail to
240
address new problems or concerns.
241
242
Each version is given a distinguishing version number.  If the Program
243
specifies a version number of this License which applies to it and "any
244
later version", you have the option of following the terms and conditions
245
either of that version or of any later version published by the Free
246
Software Foundation.  If the Program does not specify a version number of
247
this License, you may choose any version ever published by the Free Software
248
Foundation.
249
250
  10. If you wish to incorporate parts of the Program into other free
251
programs whose distribution conditions are different, write to the author
252
to ask for permission.  For software which is copyrighted by the Free
253
Software Foundation, write to the Free Software Foundation; we sometimes
254
make exceptions for this.  Our decision will be guided by the two goals
255
of preserving the free status of all derivatives of our free software and
256
of promoting the sharing and reuse of software generally.
257
258
			    NO WARRANTY
259
260
  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
261
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
262
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
263
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
264
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
265
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
266
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
267
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
268
REPAIR OR CORRECTION.
269
270
  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
271
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
272
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
273
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
274
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
275
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
276
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
277
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
278
POSSIBILITY OF SUCH DAMAGES.
279
280
		     END OF TERMS AND CONDITIONS
281
282
	    How to Apply These Terms to Your New Programs
283
284
  If you develop a new program, and you want it to be of the greatest
285
possible use to the public, the best way to achieve this is to make it
286
free software which everyone can redistribute and change under these terms.
287
288
  To do so, attach the following notices to the program.  It is safest
289
to attach them to the start of each source file to most effectively
290
convey the exclusion of warranty; and each file should have at least
291
the "copyright" line and a pointer to where the full notice is found.
292
293
    <one line to give the program's name and a brief idea of what it does.>
294
    Copyright (C) <year>  <name of author>
295
296
    This program is free software; you can redistribute it and/or modify
297
    it under the terms of the GNU General Public License as published by
298
    the Free Software Foundation; either version 2 of the License, or
299
    (at your option) any later version.
300
301
    This program is distributed in the hope that it will be useful,
302
    but WITHOUT ANY WARRANTY; without even the implied warranty of
303
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
304
    GNU General Public License for more details.
305
306
    You should have received a copy of the GNU General Public License
307
    along with this program; if not, write to the Free Software
308
    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
309
310
311
Also add information on how to contact you by electronic and paper mail.
312
313
If the program is interactive, make it output a short notice like this
314
when it starts in an interactive mode:
315
316
    Gnomovision version 69, Copyright (C) year name of author
317
    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
318
    This is free software, and you are welcome to redistribute it
319
    under certain conditions; type `show c' for details.
320
321
The hypothetical commands `show w' and `show c' should show the appropriate
322
parts of the General Public License.  Of course, the commands you use may
323
be called something other than `show w' and `show c'; they could even be
324
mouse-clicks or menu items--whatever suits your program.
325
326
You should also get your employer (if you work as a programmer) or your
327
school, if any, to sign a "copyright disclaimer" for the program, if
328
necessary.  Here is a sample; alter the names:
329
330
  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
331
  `Gnomovision' (which makes passes at compilers) written by James Hacker.
332
333
  <signature of Ty Coon>, 1 April 1989
334
  Ty Coon, President of Vice
335
336
This General Public License does not permit incorporating your program into
337
proprietary programs.  If your program is a subroutine library, you may
338
consider it more useful to permit linking proprietary applications with the
339
library.  If this is what you want to do, use the GNU Library General
340
Public License instead of this License.
(-)flickrfs-1.3.9.orig/flickrapi.py (-531 lines)
Lines 1-531 Link Here
1
#!/usr/bin/python
2
#
3
# Flickr API implementation
4
#
5
# Inspired largely by Michele Campeotto's flickrclient and Aaron Swartz'
6
# xmltramp... but I wanted to get a better idea of how python worked in
7
# those regards, so I mostly worked those components out for myself.
8
#
9
# http://micampe.it/things/flickrclient
10
# http://www.aaronsw.com/2002/xmltramp/
11
#
12
# Release 1: initial release
13
# Release 2: added upload functionality
14
# Release 3: code cleanup, convert to doc strings
15
# Release 4: better permission support
16
# Release 5: converted into fuller-featured "flickrapi"
17
# Release 6: fix upload sig bug (thanks Deepak Jois), encode test output
18
#
19
# Copyright 2005 Brian "Beej Jorgensen" Hall  beej@beej.us
20
#
21
#    This work is licensed under the Creative Commons
22
#    Attribution License.  To view a copy of this license,
23
#    visit http://creativecommons.org/licenses/by/2.5/ or send
24
#    a letter to Creative Commons, 543 Howard Street, 5th
25
#    Floor, San Francisco, California, 94105, USA.
26
#
27
# This license says that I must be credited for any derivative works.
28
# You do not need to credit me to simply use the FlickrAPI classes in
29
# your Python scripts-- you only need to credit me if you're taking this
30
# FlickrAPI class and transforming it into your own.
31
#
32
# Previous versions of this API were granted to the public domain.
33
# You're free to use those as you please.
34
#
35
# Beej Jorgensen, August 2005
36
# beej@beej.us
37
#------------------------------------------
38
# Modified(debugged portions + added functionality) by Manish Rai Jain <manishrjain@gmail.com>
39
# If you are interested in finding out exactly
40
# what portions I have modified, just search for 'manish'. Each change is 
41
# tagged with my name for easy locate. 
42
# Additional modifications similarly tagged by R. David Murray <rdmurray@bitdance.com>
43
44
import sys
45
import md5
46
import string
47
import urllib
48
import httplib
49
import os.path
50
import xml.dom.minidom
51
import urllib2
52
import socket
53
DEBUG = 0
54
########################################################################
55
# XML functionality
56
########################################################################
57
58
#-----------------------------------------------------------------------
59
class XMLNode:
60
	"""XMLNode -- generic class for holding an XML node
61
	xmlStr = \"\"\"<xml foo="32">
62
	<name bar="10">Name0</name>
63
	<name bar="11" baz="12">Name1</name>
64
	</xml>\"\"\"
65
66
	f = XMLNode.parseXML(xmlStr)
67
68
	print f.elementName
69
	print f['foo']
70
	print f.name
71
	print f.name[0].elementName
72
	print f.name[0]["bar"]
73
	print f.name[0].elementText
74
	print f.name[1].elementName
75
	print f.name[1]["bar"]
76
	print f.name[1]["baz"]
77
78
	"""
79
80
	def __init__(self):
81
		"""Construct an empty XML node."""
82
		self.elementName=""
83
		self.elementText=""
84
		self.attrib={}
85
		self.xml=""
86
87
	def __setitem__(self, key, item):
88
		"""Store a node's attribute in the attrib hash."""
89
		self.attrib[key] = item
90
91
	def __getitem__(self, key):
92
		"""Retrieve a node's attribute from the attrib hash."""
93
		return self.attrib[key]
94
95
	# Modified here: add a couple of methods to make it even easier to handle errors.
96
	# Mod by: R. David Murray <rdmurray@bitdance.com>
97
	def __nonzero__(self):
98
		if self['stat'] == "fail": return False
99
		return True
100
101
	def get_errortext(self):
102
		if self: return ''
103
		return "%s: error %s: %s\n" % (self.elementName, \
104
				self.err[0]['code'], self.err[0]['msg'])
105
	errormsg = property(get_errortext)
106
107
	#-----------------------------------------------------------------------
108
	#@classmethod
109
	def parseXML(cls, xmlStr="", storeXML=False):
110
		"""Convert an XML string into a nice instance tree of XMLNodes.
111
112
		xmlStr -- the XML to parse
113
		storeXML -- if True, stores the XML string in the root XMLNode.xml
114
115
		"""
116
117
		def __parseXMLElement(element, thisNode):
118
			"""Recursive call to process this XMLNode."""
119
			thisNode.elementName = element.nodeName
120
121
			#print element.nodeName
122
123
			# add element attributes as attributes to this node
124
			for i in range(element.attributes.length):
125
				an = element.attributes.item(i)
126
				thisNode[an.name] = an.nodeValue
127
128
			for a in element.childNodes:
129
				if a.nodeType == xml.dom.Node.ELEMENT_NODE:
130
131
					child = XMLNode()
132
					try:
133
						list = getattr(thisNode, a.nodeName)
134
					except AttributeError:
135
						setattr(thisNode, a.nodeName, [])
136
137
					# add the child node as an attrib to this node
138
					list = getattr(thisNode, a.nodeName);
139
					#print "appending child: %s to %s" % (a.nodeName, thisNode.elementName)
140
					list.append(child);
141
142
					__parseXMLElement(a, child)
143
144
				elif a.nodeType == xml.dom.Node.TEXT_NODE:
145
					thisNode.elementText += a.nodeValue
146
			
147
			return thisNode
148
		dom = xml.dom.minidom.parseString(xmlStr)
149
150
		# get the root
151
		rootNode = XMLNode()
152
		if storeXML: rootNode.xml = xmlStr
153
154
		return __parseXMLElement(dom.firstChild, rootNode)
155
156
########################################################################
157
# Flickr functionality
158
########################################################################
159
160
#-----------------------------------------------------------------------
161
class FlickrAPI:
162
	"""Encapsulated flickr functionality.
163
164
	Example usage:
165
166
	  flickr = FlickrAPI(flickrAPIKey, flickrSecret)
167
	  rsp = flickr.auth_checkToken(api_key=flickrAPIKey, auth_token=token)
168
169
	"""
170
	flickrHost = "flickr.com"
171
	flickrRESTForm = "/services/rest/"
172
	flickrAuthForm = "/services/auth/"
173
	flickrUploadForm = "/services/upload/"
174
	#-------------------------------------------------------------------
175
	def __init__(self, apiKey, secret):
176
		"""Construct a new FlickrAPI instance for a given API key and secret."""
177
		self.apiKey = apiKey
178
		self.secret = secret
179
180
		self.__handlerCache={}
181
		socket.setdefaulttimeout(10)
182
	#-------------------------------------------------------------------
183
	def __sign(self, data):
184
		"""Calculate the flickr signature for a set of params.
185
186
		data -- a hash of all the params and values to be hashed, e.g.
187
		        {"api_key":"AAAA", "auth_token":"TTTT"}
188
189
		"""
190
		dataName = ""
191
		if self.secret is not None:
192
			dataName = self.secret
193
		#print data
194
		keys = data.keys()
195
		keys.sort()
196
197
		for a in keys:
198
			if data[a] is not None:
199
				dataName += "%s%s" % (a, data[a])
200
		#print 'dataName:', dataName
201
		hash = md5.new()
202
		hash.update(dataName)
203
		return hash.hexdigest()
204
205
	#-------------------------------------------------------------------
206
	def __getattr__(self, method, **arg):
207
		"""Handle all the flickr API calls.
208
		
209
		This is Michele Campeotto's cleverness, wherein he writes a
210
		general handler for methods not defined, and assumes they are
211
		flickr methods.  He then converts them to a form to be passed as
212
		the method= parameter, and goes from there.
213
214
		http://micampe.it/things/flickrclient
215
216
		My variant is the same basic thing, except it tracks if it has
217
		already created a handler for a specific call or not.
218
219
		example usage:
220
221
			flickr.auth_getFrob(api_key="AAAAAA")
222
			rsp = flickr.favorites_getList(api_key=flickrAPIKey, \\
223
				auth_token=token)
224
225
		"""
226
227
		if not self.__handlerCache.has_key(method):
228
			def handler(_self = self, _method = method, **arg):
229
				_method = "flickr." + _method.replace("_", ".")
230
				url = "http://" + FlickrAPI.flickrHost + \
231
					FlickrAPI.flickrRESTForm
232
				arg["method"] = _method
233
				# Modified here: use default api_key and auth_token if not supplied
234
				# Mod by: R. David Murray <rdmurray@bitdance.com>
235
	
236
				#API Key is used in all the methods. So, instead of specifying it as 
237
				#parameter in calling function, we can append it over here by default.
238
				#Token eq. to Authentication is not required for ALL methods. So, 
239
				#better specify when needed. -Manish
240
241
				if not 'api_key' in arg: arg["api_key"] = _self.apiKey
242
#				if not 'auth_token' in arg and hasattr(_self, 'token'): arg['auth_token'] = _self.token
243
244
				postData = str(urllib.urlencode(arg)) + "&api_sig=" + \
245
					str(_self.__sign(arg))
246
				if DEBUG:
247
					print "--url---------------------------------------------"
248
					print url
249
					print "--postData----------------------------------------"
250
					print postData
251
				data = '<rsp stat="ok"></rsp>'
252
				req = urllib2.Request(url, postData)
253
#				socket.defaulttimetout(60)
254
				f = urllib2.urlopen(req)
255
				data = f.read()
256
				if DEBUG:
257
					print "--response----------------------------------------"
258
					print data
259
				f.close()
260
				tempNode = XMLNode()
261
				# Modified here: added XMLNode() as the 1st argument
262
				# Mod by: Manish Rai Jain <manishrjain@gmail.com>
263
				return XMLNode.parseXML(XMLNode(),xmlStr=data, storeXML=True)
264
265
			self.__handlerCache[method] = handler;
266
267
		return self.__handlerCache[method]
268
	
269
	#-------------------------------------------------------------------
270
	def __getAuthURL(self, perms, frob):
271
		"""Return the authorization URL to get a token.
272
273
		This is the URL the app will launch a browser toward if it
274
		needs a new token.
275
			
276
		perms -- "read", "write", or "delete"
277
		frob -- picked up from an earlier call to FlickrAPI.auth_getFrob()
278
279
		"""
280
281
		data = {"api_key": self.apiKey, "frob": frob, "perms": perms}
282
		data["api_sig"] = self.__sign(data)
283
		return "http://%s%s?%s" % (FlickrAPI.flickrHost, \
284
			FlickrAPI.flickrAuthForm, urllib.urlencode(data))
285
286
	#-------------------------------------------------------------------
287
	def upload(self, filename, jpegData="", **arg):
288
		"""Upload a file to flickr.
289
290
		jpegData -- send buffered data read from file instead of filename
291
		
292
		Be extra careful you spell the parameters correctly, or you will
293
		get a rather cryptic "Invalid Signature" error on the upload!
294
295
		Supported parameters:
296
297
		api_key
298
		auth_token -- documentation mistakenly calls this "auth_hash"
299
		title
300
		description
301
		tags -- space-delimited list of tags, "tag1 tag2 tag3"
302
		is_public -- "1" or "0"
303
		is_friend -- "1" or "0"
304
		is_family -- "1" or "0"
305
306
		"""
307
308
		# verify key names
309
		for a in arg.keys():
310
			if a != "api_key" and a != "auth_token" and a != "title" and \
311
				a != "description" and a != "tags" and a != "is_public" and \
312
				a != "is_friend" and a != "is_family":
313
314
				sys.stderr.write("FlickrAPI: warning: unknown parameter " \
315
					"\"%s\" sent to FlickrAPI.upload\n" % (a))
316
		
317
		# Modified here: use default api_key and auth_token if not supplied
318
		# Mod by: R. David Murray <rdmurray@bitdance.com>
319
		if not 'api_key' in arg: arg["api_key"] = self.apiKey
320
		if not 'auth_token' in arg: arg['auth_token'] = self.token
321
		arg["api_sig"] = self.__sign(arg)
322
		url = "http://" + FlickrAPI.flickrHost + FlickrAPI.flickrUploadForm
323
324
		# construct POST data
325
#		boundary = "===beej=jorgensen==========7d45e178b0434"
326
		import mimetools
327
		boundary = mimetools.choose_boundary()
328
		Hdr = "multipart/form-data; boundary=%s" % boundary
329
		body = ""
330
331
		# required params
332
		for a in ('api_key', 'auth_token', 'api_sig'):
333
			body += "--%s\r\n" % (boundary)
334
			body += "Content-Disposition: form-data; name=\""+a+"\"\r\n\r\n"
335
			body += "%s\r\n" % (arg[a])
336
337
		# optional params
338
		for a in ('title', 'description', 'tags', 'is_public', \
339
			'is_friend', 'is_family'):
340
341
			if arg.has_key(a):
342
				body += "--%s\r\n" % (boundary)
343
				body += "Content-Disposition: form-data; name=\""+a+"\"\r\n\r\n"
344
				body += "%s\r\n" % (arg[a])
345
346
		body += "--%s\r\n" % (boundary)
347
		body += "Content-Disposition: form-data; name=\"photo\";"
348
		body += " filename=\"%s\"\r\n" % filename
349
		body += "Content-Type: image/jpeg\r\n\r\n"
350
351
		#print body
352
353
		try:
354
		# Added by Manish <manishrjain@gmail.com> for allowing upload
355
		# by sending buffer instead of specifying filename
356
			if jpegData=="":
357
				fp = file(filename, "rb")
358
				jpegData = fp.read()
359
				fp.close()
360
361
			postData = body.encode("utf_8") + jpegData + "\r\n" + \
362
				("--%s--" % (boundary)).encode("utf_8")
363
		
364
		
365
		# Modified by Manish <manishrjain@gmail.com> for allowing
366
		# upload through proxy
367
		
368
			request = urllib2.Request(url)
369
			request.add_data(postData)
370
			request.add_header("Content-Type", Hdr)
371
			response = urllib2.urlopen(request)
372
			rspXML = response.read()
373
			# Modified by Manish Rai Jain <manishrjain@gmail.com>
374
			return XMLNode.parseXML(XMLNode(), xmlStr=rspXML)
375
		except IOError:
376
			return None
377
378
379
380
	#-----------------------------------------------------------------------
381
	#@classmethod
382
	def testFailure(cls, rsp, exit=True):
383
		"""Exit app if the rsp XMLNode indicates failure."""
384
		if rsp['stat'] == "fail":
385
			sys.stderr.write("%s: error %s: %s\n" % (rsp.elementName, \
386
				rsp.err[0]['code'], rsp.err[0]['msg']))
387
			if exit: sys.exit(1)
388
389
	#-----------------------------------------------------------------------
390
	def __getCachedTokenPath(self):
391
		"""Return the directory holding the app data."""
392
		return os.path.expanduser("~/.flickr/%s" % (self.apiKey))
393
394
	#-----------------------------------------------------------------------
395
	def __getCachedTokenFilename(self):
396
		"""Return the full pathname of the cached token file."""
397
		return "%s/auth.xml" % (self.__getCachedTokenPath())
398
399
	#-----------------------------------------------------------------------
400
	def __getCachedToken(self):
401
		"""Read and return a cached token, or None if not found.
402
403
		The token is read from the cached token file, which is basically the
404
		entire RSP response containing the auth element.
405
		"""
406
407
		try:
408
			f = file(self.__getCachedTokenFilename(), "r")
409
			data = f.read()
410
			f.close()
411
			# Modified here: added XMLNode() as the 1st argument
412
			# Mod by: Manish Rai Jain <manishrjain@gmail.com>
413
			rsp = XMLNode.parseXML(XMLNode(), data)
414
415
			return rsp.auth[0].token[0].elementText
416
417
		except IOError:
418
			return None
419
420
	#-----------------------------------------------------------------------
421
	def __setCachedToken(self, xml):
422
		"""Cache a token for later use.
423
424
		The cached tag is stored by simply saving the entire RSP response
425
		containing the auth element.
426
427
		"""
428
429
		path = self.__getCachedTokenPath()
430
		if not os.path.exists(path):
431
			os.makedirs(path)
432
433
		f = file(self.__getCachedTokenFilename(), "w")
434
		f.write(xml)
435
		f.close()
436
437
438
	#-----------------------------------------------------------------------
439
	def getToken(self, perms="write", browser="lynx"):
440
		"""Get a token either from the cache, or make a new one from the
441
		frob.
442
443
		This first attempts to find a token in the user's token cache on
444
		disk.
445
		
446
		If that fails (or if the token is no longer valid based on
447
		flickr.auth.checkToken) a new frob is acquired.  The frob is
448
		validated by having the user log into flickr (with lynx), and
449
		subsequently a valid token is retrieved.
450
451
		The newly minted token is then cached locally for the next run.
452
453
		perms--"read", "write", or "delete"
454
		browser--whatever browser should be used in the system() call
455
456
		"""
457
		
458
		# see if we have a saved token
459
		token = self.__getCachedToken()
460
461
		# see if it's valid
462
		if token != None:
463
			rsp = self.auth_checkToken(api_key=self.apiKey, auth_token=token)
464
			if rsp['stat'] != "ok":
465
				token = None
466
			else:
467
				# see if we have enough permissions
468
				tokenPerms = rsp.auth[0].perms[0].elementText
469
				if tokenPerms == "read" and perms != "read": token = None
470
				elif tokenPerms == "write" and perms == "delete": token = None
471
472
		# get a new token if we need one
473
		if token == None:
474
			# get the frob
475
			rsp = self.auth_getFrob(api_key=self.apiKey)
476
			self.testFailure(rsp)
477
			frob = rsp.frob[0].elementText
478
479
			# validate online
480
			os.system("%s '%s'" % (browser, self.__getAuthURL(perms, frob)))
481
			# get a token
482
			rsp = self.auth_getToken(api_key=self.apiKey, frob=frob)
483
			self.testFailure(rsp)
484
			token = rsp.auth[0].token[0].elementText
485
			
486
			# store the auth info for next time
487
			self.__setCachedToken(rsp.xml)
488
489
		# Modified here: save the auth token to use as a default
490
		# Mod by: R. David Murray <rdmurray@bitdance.com>
491
		self.token = token
492
		return token
493
494
495
########################################################################
496
# App functionality
497
########################################################################
498
499
def main(argv):
500
	# flickr auth information:
501
	flickrAPIKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"  # API key
502
	flickrSecret = "yyyyyyyyyyyyyyyy"                  # shared "secret"
503
504
	# make a new FlickrAPI instance
505
	fapi = FlickrAPI(flickrAPIKey, flickrSecret)
506
507
	# do the whole whatever-it-takes to get a valid token:
508
	token = fapi.getToken(browser="/opt/firefox/firefox")
509
510
	# get my favorites
511
	rsp = fapi.favorites_getList(api_key=flickrAPIKey,auth_token=token)
512
	fapi.testFailure(rsp)
513
514
	# and print them
515
	for a in rsp.photos[0].photo:
516
		print "%10s: %s" % (a['id'], a['title'].encode("ascii", "replace"))
517
518
	# upload the file foo.jpg
519
	#rsp = fapi.upload("foo.jpg", api_key=flickrAPIKey, auth_token=token, \
520
	#	title="This is the title", description="This is the description", \
521
	#	tags="tag1 tag2 tag3", is_public="1")
522
	#if rsp == None:
523
	#	sys.stderr.write("can't find file\n")
524
	#else:
525
	#	fapi.testFailure(rsp)
526
527
	return 0
528
529
# run the main if we're not being imported:
530
if __name__ == "__main__": sys.exit(main(sys.argv))
531
(-)flickrfs-1.3.9.orig/flickrfs/flickrapi.py (+549 lines)
Line 0 Link Here
1
# Flickr API implementation
2
#
3
# Inspired largely by Michele Campeotto's flickrclient and Aaron Swartz'
4
# xmltramp... but I wanted to get a better idea of how python worked in
5
# those regards, so I mostly worked those components out for myself.
6
#
7
# http://micampe.it/things/flickrclient
8
# http://www.aaronsw.com/2002/xmltramp/
9
#
10
# Release 1: initial release
11
# Release 2: added upload functionality
12
# Release 3: code cleanup, convert to doc strings
13
# Release 4: better permission support
14
# Release 5: converted into fuller-featured "flickrapi"
15
# Release 6: fix upload sig bug (thanks Deepak Jois), encode test output
16
# Release 7: fix path construction, Manish Rai Jain's improvements, exceptions
17
# Release 8: change API endpoint to "api.flickr.com"
18
# Release 9: change to MIT license
19
# Release 10: fix horrid \r\n bug on final boundary
20
# Release 11: break out validateFrob() for subclassing
21
#
22
# Work by (or inspired by) Manish Rai Jain <manishrjain@gmail.com>:
23
#
24
#    improved error reporting, proper multipart MIME boundary creation,
25
#    use of urllib2 to allow uploads through a proxy, upload accepts
26
#    raw data as well as a filename
27
#
28
# Copyright (c) 2007 Brian "Beej Jorgensen" Hall
29
#
30
# Permission is hereby granted, free of charge, to any person obtaining
31
# a copy of this software and associated documentation files (the
32
# "Software"), to deal in the Software without restriction, including
33
# without limitation the rights to use, copy, modify, merge, publish,
34
# distribute, sublicense, and/or sell copies of the Software, and to
35
# permit persons to whom the Software is furnished to do so, subject to
36
# the following conditions:
37
#
38
# The above copyright notice and this permission notice shall be
39
# included in all copies or substantial portions of the Software.
40
#
41
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
42
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
43
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
44
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
45
# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
46
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
47
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
48
#
49
# Certain previous versions of this API were granted to the public
50
# domain.  You're free to use those as you please.
51
#
52
# Beej Jorgensen, Maintainer, 19-Jan-2007
53
# beej@beej.us
54
#
55
#------------------------------------------
56
# Modified(debugged portions + added functionality) by Manish Rai Jain <manishrjain@gmail.com>
57
# If you are interested in finding out exactly
58
# what portions I have modified, just search for 'manish'. Each change is 
59
# tagged with my name for easy locate. 
60
# Additional modifications similarly tagged by R. David Murray <rdmurray@bitdance.com>
61
62
import sys
63
import md5
64
import string
65
import urllib
66
import httplib
67
import os.path
68
import xml.dom.minidom
69
import urllib2
70
import socket
71
DEBUG = 0
72
########################################################################
73
# XML functionality
74
########################################################################
75
76
#-----------------------------------------------------------------------
77
class XMLNode:
78
	"""XMLNode -- generic class for holding an XML node
79
	xmlStr = \"\"\"<xml foo="32">
80
	<name bar="10">Name0</name>
81
	<name bar="11" baz="12">Name1</name>
82
	</xml>\"\"\"
83
84
	f = XMLNode.parseXML(xmlStr)
85
86
	print f.elementName
87
	print f['foo']
88
	print f.name
89
	print f.name[0].elementName
90
	print f.name[0]["bar"]
91
	print f.name[0].elementText
92
	print f.name[1].elementName
93
	print f.name[1]["bar"]
94
	print f.name[1]["baz"]
95
96
	"""
97
98
	def __init__(self):
99
		"""Construct an empty XML node."""
100
		self.elementName=""
101
		self.elementText=""
102
		self.attrib={}
103
		self.xml=""
104
105
	def __setitem__(self, key, item):
106
		"""Store a node's attribute in the attrib hash."""
107
		self.attrib[key] = item
108
109
	def __getitem__(self, key):
110
		"""Retrieve a node's attribute from the attrib hash."""
111
		return self.attrib[key]
112
113
	# Modified here: add a couple of methods to make it even easier to handle errors.
114
	# Mod by: R. David Murray <rdmurray@bitdance.com>
115
	def __nonzero__(self):
116
		if self['stat'] == "fail": return False
117
		return True
118
119
	def get_errortext(self):
120
		if self: return ''
121
		return "%s: error %s: %s\n" % (self.elementName, \
122
				self.err[0]['code'], self.err[0]['msg'])
123
	errormsg = property(get_errortext)
124
125
	#-----------------------------------------------------------------------
126
	#@classmethod
127
	def parseXML(cls, xmlStr="", storeXML=False):
128
		"""Convert an XML string into a nice instance tree of XMLNodes.
129
130
		xmlStr -- the XML to parse
131
		storeXML -- if True, stores the XML string in the root XMLNode.xml
132
133
		"""
134
135
		def __parseXMLElement(element, thisNode):
136
			"""Recursive call to process this XMLNode."""
137
			thisNode.elementName = element.nodeName
138
139
			#print element.nodeName
140
141
			# add element attributes as attributes to this node
142
			for i in range(element.attributes.length):
143
				an = element.attributes.item(i)
144
				thisNode[an.name] = an.nodeValue
145
146
			for a in element.childNodes:
147
				if a.nodeType == xml.dom.Node.ELEMENT_NODE:
148
149
					child = XMLNode()
150
					try:
151
						list = getattr(thisNode, a.nodeName)
152
					except AttributeError:
153
						setattr(thisNode, a.nodeName, [])
154
155
					# add the child node as an attrib to this node
156
					list = getattr(thisNode, a.nodeName);
157
					#print "appending child: %s to %s" % (a.nodeName, thisNode.elementName)
158
					list.append(child);
159
160
					__parseXMLElement(a, child)
161
162
				elif a.nodeType == xml.dom.Node.TEXT_NODE:
163
					thisNode.elementText += a.nodeValue
164
			
165
			return thisNode
166
		dom = xml.dom.minidom.parseString(xmlStr)
167
168
		# get the root
169
		rootNode = XMLNode()
170
		if storeXML: rootNode.xml = xmlStr
171
172
		return __parseXMLElement(dom.firstChild, rootNode)
173
174
########################################################################
175
# Flickr functionality
176
########################################################################
177
178
#-----------------------------------------------------------------------
179
class FlickrAPI:
180
	"""Encapsulated flickr functionality.
181
182
	Example usage:
183
184
	  flickr = FlickrAPI(flickrAPIKey, flickrSecret)
185
	  rsp = flickr.auth_checkToken(api_key=flickrAPIKey, auth_token=token)
186
187
	"""
188
	flickrHost = "flickr.com"
189
	flickrRESTForm = "/services/rest/"
190
	flickrAuthForm = "/services/auth/"
191
	flickrUploadForm = "/services/upload/"
192
	#-------------------------------------------------------------------
193
	def __init__(self, apiKey, secret):
194
		"""Construct a new FlickrAPI instance for a given API key and secret."""
195
		self.apiKey = apiKey
196
		self.secret = secret
197
198
		self.__handlerCache={}
199
		socket.setdefaulttimeout(10)
200
	#-------------------------------------------------------------------
201
	def __sign(self, data):
202
		"""Calculate the flickr signature for a set of params.
203
204
		data -- a hash of all the params and values to be hashed, e.g.
205
		        {"api_key":"AAAA", "auth_token":"TTTT"}
206
207
		"""
208
		dataName = ""
209
		if self.secret is not None:
210
			dataName = self.secret
211
		#print data
212
		keys = data.keys()
213
		keys.sort()
214
215
		for a in keys:
216
			if data[a] is not None:
217
				dataName += "%s%s" % (a, data[a])
218
		#print 'dataName:', dataName
219
		hash = md5.new()
220
		hash.update(dataName)
221
		return hash.hexdigest()
222
223
	#-------------------------------------------------------------------
224
	def __getattr__(self, method, **arg):
225
		"""Handle all the flickr API calls.
226
		
227
		This is Michele Campeotto's cleverness, wherein he writes a
228
		general handler for methods not defined, and assumes they are
229
		flickr methods.  He then converts them to a form to be passed as
230
		the method= parameter, and goes from there.
231
232
		http://micampe.it/things/flickrclient
233
234
		My variant is the same basic thing, except it tracks if it has
235
		already created a handler for a specific call or not.
236
237
		example usage:
238
239
			flickr.auth_getFrob(api_key="AAAAAA")
240
			rsp = flickr.favorites_getList(api_key=flickrAPIKey, \\
241
				auth_token=token)
242
243
		"""
244
245
		if not self.__handlerCache.has_key(method):
246
			def handler(_self = self, _method = method, **arg):
247
				_method = "flickr." + _method.replace("_", ".")
248
				url = "http://" + FlickrAPI.flickrHost + \
249
					FlickrAPI.flickrRESTForm
250
				arg["method"] = _method
251
				# Modified here: use default api_key and auth_token if not supplied
252
				# Mod by: R. David Murray <rdmurray@bitdance.com>
253
	
254
				#API Key is used in all the methods. So, instead of specifying it as 
255
				#parameter in calling function, we can append it over here by default.
256
				#Token eq. to Authentication is not required for ALL methods. So, 
257
				#better specify when needed. -Manish
258
259
				if not 'api_key' in arg: arg["api_key"] = _self.apiKey
260
#				if not 'auth_token' in arg and hasattr(_self, 'token'): arg['auth_token'] = _self.token
261
262
				postData = str(urllib.urlencode(arg)) + "&api_sig=" + \
263
					str(_self.__sign(arg))
264
				if DEBUG:
265
					print "--url---------------------------------------------"
266
					print url
267
					print "--postData----------------------------------------"
268
					print postData
269
				data = '<rsp stat="ok"></rsp>'
270
				req = urllib2.Request(url, postData)
271
#				socket.defaulttimetout(60)
272
				f = urllib2.urlopen(req)
273
				data = f.read()
274
				if DEBUG:
275
					print "--response----------------------------------------"
276
					print data
277
				f.close()
278
				tempNode = XMLNode()
279
				# Modified here: added XMLNode() as the 1st argument
280
				# Mod by: Manish Rai Jain <manishrjain@gmail.com>
281
				return XMLNode.parseXML(XMLNode(),xmlStr=data, storeXML=True)
282
283
			self.__handlerCache[method] = handler;
284
285
		return self.__handlerCache[method]
286
	
287
	#-------------------------------------------------------------------
288
	def __getAuthURL(self, perms, frob):
289
		"""Return the authorization URL to get a token.
290
291
		This is the URL the app will launch a browser toward if it
292
		needs a new token.
293
			
294
		perms -- "read", "write", or "delete"
295
		frob -- picked up from an earlier call to FlickrAPI.auth_getFrob()
296
297
		"""
298
299
		data = {"api_key": self.apiKey, "frob": frob, "perms": perms}
300
		data["api_sig"] = self.__sign(data)
301
		return "http://%s%s?%s" % (FlickrAPI.flickrHost, \
302
			FlickrAPI.flickrAuthForm, urllib.urlencode(data))
303
304
	#-------------------------------------------------------------------
305
	def upload(self, filename, jpegData="", **arg):
306
		"""Upload a file to flickr.
307
308
		jpegData -- send buffered data read from file instead of filename
309
		
310
		Be extra careful you spell the parameters correctly, or you will
311
		get a rather cryptic "Invalid Signature" error on the upload!
312
313
		Supported parameters:
314
315
		api_key
316
		auth_token -- documentation mistakenly calls this "auth_hash"
317
		title
318
		description
319
		tags -- space-delimited list of tags, "tag1 tag2 tag3"
320
		is_public -- "1" or "0"
321
		is_friend -- "1" or "0"
322
		is_family -- "1" or "0"
323
324
		"""
325
326
		# verify key names
327
		for a in arg.keys():
328
			if a != "api_key" and a != "auth_token" and a != "title" and \
329
				a != "description" and a != "tags" and a != "is_public" and \
330
				a != "is_friend" and a != "is_family":
331
332
				sys.stderr.write("FlickrAPI: warning: unknown parameter " \
333
					"\"%s\" sent to FlickrAPI.upload\n" % (a))
334
		
335
		# Modified here: use default api_key and auth_token if not supplied
336
		# Mod by: R. David Murray <rdmurray@bitdance.com>
337
		if not 'api_key' in arg: arg["api_key"] = self.apiKey
338
		if not 'auth_token' in arg: arg['auth_token'] = self.token
339
		arg["api_sig"] = self.__sign(arg)
340
		url = "http://" + FlickrAPI.flickrHost + FlickrAPI.flickrUploadForm
341
342
		# construct POST data
343
#		boundary = "===beej=jorgensen==========7d45e178b0434"
344
		import mimetools
345
		boundary = mimetools.choose_boundary()
346
		Hdr = "multipart/form-data; boundary=%s" % boundary
347
		body = ""
348
349
		# required params
350
		for a in ('api_key', 'auth_token', 'api_sig'):
351
			body += "--%s\r\n" % (boundary)
352
			body += "Content-Disposition: form-data; name=\""+a+"\"\r\n\r\n"
353
			body += "%s\r\n" % (arg[a])
354
355
		# optional params
356
		for a in ('title', 'description', 'tags', 'is_public', \
357
			'is_friend', 'is_family'):
358
359
			if arg.has_key(a):
360
				body += "--%s\r\n" % (boundary)
361
				body += "Content-Disposition: form-data; name=\""+a+"\"\r\n\r\n"
362
				body += "%s\r\n" % (arg[a])
363
364
		body += "--%s\r\n" % (boundary)
365
		body += "Content-Disposition: form-data; name=\"photo\";"
366
		body += " filename=\"%s\"\r\n" % filename
367
		body += "Content-Type: image/jpeg\r\n\r\n"
368
369
		#print body
370
371
		try:
372
		# Added by Manish <manishrjain@gmail.com> for allowing upload
373
		# by sending buffer instead of specifying filename
374
			if jpegData=="":
375
				fp = file(filename, "rb")
376
				jpegData = fp.read()
377
				fp.close()
378
379
			postData = body.encode("utf_8") + jpegData + "\r\n" + \
380
				("--%s--" % (boundary)).encode("utf_8")
381
		
382
		
383
		# Modified by Manish <manishrjain@gmail.com> for allowing
384
		# upload through proxy
385
		
386
			request = urllib2.Request(url)
387
			request.add_data(postData)
388
			request.add_header("Content-Type", Hdr)
389
			response = urllib2.urlopen(request)
390
			rspXML = response.read()
391
			# Modified by Manish Rai Jain <manishrjain@gmail.com>
392
			return XMLNode.parseXML(XMLNode(), xmlStr=rspXML)
393
		except IOError:
394
			return None
395
396
397
398
	#-----------------------------------------------------------------------
399
	#@classmethod
400
	def testFailure(cls, rsp, exit=True):
401
		"""Exit app if the rsp XMLNode indicates failure."""
402
		if rsp['stat'] == "fail":
403
			sys.stderr.write("%s: error %s: %s\n" % (rsp.elementName, \
404
				rsp.err[0]['code'], rsp.err[0]['msg']))
405
			if exit: sys.exit(1)
406
407
	#-----------------------------------------------------------------------
408
	def __getCachedTokenPath(self):
409
		"""Return the directory holding the app data."""
410
		return os.path.expanduser("~/.flickr/%s" % (self.apiKey))
411
412
	#-----------------------------------------------------------------------
413
	def __getCachedTokenFilename(self):
414
		"""Return the full pathname of the cached token file."""
415
		return "%s/auth.xml" % (self.__getCachedTokenPath())
416
417
	#-----------------------------------------------------------------------
418
	def __getCachedToken(self):
419
		"""Read and return a cached token, or None if not found.
420
421
		The token is read from the cached token file, which is basically the
422
		entire RSP response containing the auth element.
423
		"""
424
425
		try:
426
			f = file(self.__getCachedTokenFilename(), "r")
427
			data = f.read()
428
			f.close()
429
			# Modified here: added XMLNode() as the 1st argument
430
			# Mod by: Manish Rai Jain <manishrjain@gmail.com>
431
			rsp = XMLNode.parseXML(XMLNode(), data)
432
433
			return rsp.auth[0].token[0].elementText
434
435
		except IOError:
436
			return None
437
438
	#-----------------------------------------------------------------------
439
	def __setCachedToken(self, xml):
440
		"""Cache a token for later use.
441
442
		The cached tag is stored by simply saving the entire RSP response
443
		containing the auth element.
444
445
		"""
446
447
		path = self.__getCachedTokenPath()
448
		if not os.path.exists(path):
449
			os.makedirs(path)
450
451
		f = file(self.__getCachedTokenFilename(), "w")
452
		f.write(xml)
453
		f.close()
454
455
456
	#-----------------------------------------------------------------------
457
	def getToken(self, perms="write", browser="lynx"):
458
		"""Get a token either from the cache, or make a new one from the
459
		frob.
460
461
		This first attempts to find a token in the user's token cache on
462
		disk.
463
		
464
		If that fails (or if the token is no longer valid based on
465
		flickr.auth.checkToken) a new frob is acquired.  The frob is
466
		validated by having the user log into flickr (with lynx), and
467
		subsequently a valid token is retrieved.
468
469
		The newly minted token is then cached locally for the next run.
470
471
		perms--"read", "write", or "delete"
472
		browser--whatever browser should be used in the system() call
473
474
		"""
475
		
476
		# see if we have a saved token
477
		token = self.__getCachedToken()
478
479
		# see if it's valid
480
		if token != None:
481
			rsp = self.auth_checkToken(api_key=self.apiKey, auth_token=token)
482
			if rsp['stat'] != "ok":
483
				token = None
484
			else:
485
				# see if we have enough permissions
486
				tokenPerms = rsp.auth[0].perms[0].elementText
487
				if tokenPerms == "read" and perms != "read": token = None
488
				elif tokenPerms == "write" and perms == "delete": token = None
489
490
		# get a new token if we need one
491
		if token == None:
492
			# get the frob
493
			rsp = self.auth_getFrob(api_key=self.apiKey)
494
			self.testFailure(rsp)
495
			frob = rsp.frob[0].elementText
496
497
			# validate online
498
			os.system("%s '%s'" % (browser, self.__getAuthURL(perms, frob)))
499
			# get a token
500
			rsp = self.auth_getToken(api_key=self.apiKey, frob=frob)
501
			self.testFailure(rsp)
502
			token = rsp.auth[0].token[0].elementText
503
			
504
			# store the auth info for next time
505
			self.__setCachedToken(rsp.xml)
506
507
		# Modified here: save the auth token to use as a default
508
		# Mod by: R. David Murray <rdmurray@bitdance.com>
509
		self.token = token
510
		return token
511
512
513
########################################################################
514
# App functionality
515
########################################################################
516
517
def main(argv):
518
	# flickr auth information:
519
	flickrAPIKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"  # API key
520
	flickrSecret = "yyyyyyyyyyyyyyyy"                  # shared "secret"
521
522
	# make a new FlickrAPI instance
523
	fapi = FlickrAPI(flickrAPIKey, flickrSecret)
524
525
	# do the whole whatever-it-takes to get a valid token:
526
	token = fapi.getToken(browser="/usr/bin/x-www-browser")
527
528
	# get my favorites
529
	rsp = fapi.favorites_getList(api_key=flickrAPIKey,auth_token=token)
530
	fapi.testFailure(rsp)
531
532
	# and print them
533
	for a in rsp.photos[0].photo:
534
		print "%10s: %s" % (a['id'], a['title'].encode("ascii", "replace"))
535
536
	# upload the file foo.jpg
537
	#rsp = fapi.upload("foo.jpg", api_key=flickrAPIKey, auth_token=token, \
538
	#	title="This is the title", description="This is the description", \
539
	#	tags="tag1 tag2 tag3", is_public="1")
540
	#if rsp == None:
541
	#	sys.stderr.write("can't find file\n")
542
	#else:
543
	#	fapi.testFailure(rsp)
544
545
	return 0
546
547
# run the main if we're not being imported:
548
if __name__ == "__main__": sys.exit(main(sys.argv))
549
(-)flickrfs-1.3.9.orig/flickrfs/flickrfs.py (+1108 lines)
Line 0 Link Here
1
#!/usr/bin/python
2
#===============================================================================
3
#  flickrfs - Virtual Filesystem for Flickr
4
#  Copyright (c) 2005,2006 Manish Rai Jain  <manishrjain@gmail.com>
5
#
6
#  This program can be distributed under the terms of the GNU GPL version 2, or 
7
#  its later versions. 
8
#
9
# DISCLAIMER: The API Key and Shared Secret are provided by the author in 
10
# the hope that it will prevent unnecessary trouble to the end-user. The 
11
# author will not be liable for any misuse of this API Key/Shared Secret 
12
# through this application/derived apps/any 3rd party apps using this key. 
13
#===============================================================================
14
15
__author__ =  "Manish Rai Jain (manishrjain@gmail.com)"
16
__license__ = "GPLv2 (details at http://www.gnu.org/licenses/licenses.html#GPL)"
17
18
import thread, string, ConfigParser, mimetypes, codecs
19
import time, logging, logging.handlers, os, sys
20
from glob import glob
21
from errno import *
22
from traceback import format_exc
23
# The python-fuse api has changed in 0.2, but we're still using the 0.1 api.
24
# The following two lines line will make us compatible with both versions by
25
# making 2.0 use the 1.0 api.
26
# (See http://fuse4bsd.creo.hu/README.new_fusepy_api.html)
27
import fuse
28
fuse.fuse_python_api = (0, 1)
29
Fuse = fuse.Fuse
30
import threading
31
import random, commands
32
from urllib2 import URLError
33
from transactions import TransFlickr
34
import inodes
35
36
#Some global definitions and functions
37
NUMRETRIES = 3
38
DEFAULTCONFIG = """\
39
[configuration]
40
41
browser: /usr/bin/x-www-browser
42
image.size:
43
sets.sync.int: 300
44
stream.sync.int: 300
45
"""
46
47
#Set up the .flickfs directory.
48
homedir = os.getenv('HOME')
49
flickrfsHome = os.path.join(homedir, '.flickrfs')
50
dbPath = os.path.join(flickrfsHome, '.inode.bdb')
51
52
if not os.path.exists(flickrfsHome):
53
  os.mkdir(os.path.join(flickrfsHome))
54
else:
55
  # Remove previous metadata files from ~/.flickrfs
56
  for a in glob(os.path.join(flickrfsHome, '.*')):
57
    os.remove(os.path.join(flickrfsHome, a))
58
  try:
59
    os.remove(dbPath)
60
  except:
61
    pass
62
 
63
# Added by Varun Hiremath
64
if not os.path.exists(flickrfsHome + "/config.txt"):
65
  fconfig = open(flickrfsHome+"/config.txt",'w')
66
  fconfig.write(DEFAULTCONFIG)
67
  fconfig.close()
68
69
# Set up logging
70
rootlogger = logging.getLogger()
71
loghdlr = logging.handlers.RotatingFileHandler(
72
                             os.path.join(flickrfsHome,'log'), "a", 5242880, 3)
73
logfmt = logging.Formatter("%(asctime)s %(name)-14s %(levelname)-7s %(threadName)-10s %(funcName)-22s %(message)s", "%x %X")
74
loghdlr.setFormatter(logfmt)
75
rootlogger.addHandler(loghdlr)
76
rootlogger.setLevel(logging.DEBUG)
77
log = logging.getLogger('flickrfs')
78
logattr = logging.getLogger('flickrfs.attr')
79
logattr.setLevel(logging.INFO)
80
81
cp = ConfigParser.ConfigParser()
82
cp.read(flickrfsHome + '/config.txt')
83
_resizeStr = ""
84
sets_sync_int = 600.0
85
stream_sync_int = 600.0
86
try:
87
  _resizeStr = cp.get('configuration', 'image.size')
88
except:
89
  print 'No default size of image found. Will upload original size of images.'
90
try:
91
  sets_sync_int = float(cp.get('configuration', 'sets.sync.int'))
92
except:
93
  pass
94
try:
95
  stream_sync_int = float(cp.get('configuration', 'stream.sync.int'))
96
except:
97
  pass
98
try:
99
  browserName = cp.get('configuration', 'browser')
100
except:
101
  pass
102
103
# Retrive the resize string.
104
def GetResizeStr():
105
  return _resizeStr
106
107
#Utility functions.
108
def _log_exception_wrapper(func, *args, **kw):
109
  """Call 'func' with args and kws and log any exception it throws.
110
  """
111
  for i in range(0, NUMRETRIES):
112
    log.debug("retry attempt %s for func %s", i, func.__name__)
113
    try:
114
      func(*args, **kw)
115
      return
116
    except:
117
      log.error("exception in function %s", func.__name__)
118
      log.error(format_exc())
119
120
def background(func, *args, **kw):
121
    """Run 'func' as a thread, logging any exceptions it throws.
122
123
    To run
124
125
      somefunc(arg1, arg2='value')
126
127
    as a thread, do:
128
129
      background(somefunc, arg1, arg2='value')
130
131
    Any exceptions thrown are logged as errors, and the traceback is logged.
132
    """
133
    thread.start_new_thread(_log_exception_wrapper, (func,)+args, kw)
134
135
def timerThread(func, func1, interval):
136
  '''Execute func now, followed by func1 every interval seconds
137
  '''
138
  log.debug("running first pass funtion %s", func.__name__)
139
  t = threading.Timer(0.0, func)
140
  try:
141
    t.run()
142
  except: 
143
    log.debug(format_exc())
144
  while(interval):
145
    log.debug("scheduling function %s to run in %s seconds",
146
        func1.__name__, interval)
147
    t = threading.Timer(interval, func1)
148
    try:
149
      t.run()
150
    except:
151
      log.debug(format_exc())
152
153
def retryFlickrOp(isNone, func, *args):
154
  # This function helps in retrying the flickr transactions, in case they fail.
155
  result = None
156
  for i in range(0, NUMRETRIES):
157
    log.debug("retry attempt %d for func %s", i, func.__name__)
158
    try:
159
      result = func(*args)
160
      if result is None:
161
        if isNone:
162
          return result
163
        else:
164
          continue
165
      else:
166
        return result
167
    except URLError, detail:
168
      log.error("URLError in function %s with error: %s",
169
          func.__name__, detail)
170
  # We've utilized all our attempts, send out the result whatever it is.
171
  return result
172
173
class Flickrfs(Fuse):
174
175
  def __init__(self, *args, **kw):
176
  
177
    Fuse.__init__(self, *args, **kw)
178
    log.info("mountpoint: %s", repr(self.mountpoint))
179
    log.info("mount options: %s", ', '.join(self.optlist))
180
    log.info("named mount options: %s",
181
        ', '.join([ "%s: %s" % (k, v) for k, v in self.optdict.items() ]))
182
    
183
    self.inodeCache = inodes.InodeCache(dbPath) # Inodes need to be stored.
184
    self.imgCache = inodes.ImageCache()
185
    self.NSID = ""
186
    self.transfl = TransFlickr(browserName)
187
188
    # Set some variables to be utilized by statfs function.
189
    self.statfsCounter = -1
190
    self.max = 0L
191
    self.used = 0L
192
193
    self.NSID = self.transfl.getUserId()
194
    if self.NSID is None:
195
      log.error("can't retrieve user information")
196
      sys.exit(-1)
197
198
    log.info('getting list of licenses available')
199
    self.licenses = self.transfl.getLicenses()
200
    if self.licenses is None:
201
      log.error("can't retrieve license information")
202
      sys.exit(-1)
203
204
    # do stuff to set up your filesystem here, if you want
205
    self._mkdir("/")
206
    self._mkdir("/tags")
207
    self._mkdir("/tags/personal")
208
    self._mkdir("/tags/public")
209
    background(timerThread, self.sets_thread, 
210
               self.sync_sets_thread, sets_sync_int) #sync every 2 minutes
211
212
213
  def imageResize(self, bufData):
214
    # If no resizing information is present, then return the buffer directly.
215
    if GetResizeStr() == "":
216
      return bufData
217
218
    # Else go ahead and do the conversion.
219
    im = '/tmp/flickrfs-' + str(int(random.random()*1000000000))
220
    f = open(im, 'w')
221
    f.write(bufData)
222
    f.close()
223
    cmd = 'identify -format "%%w" %s'%(im,)
224
    status,ret = commands.getstatusoutput(cmd)
225
    msg = ("%s command not found; you must install Imagemagick to get "
226
      "auto photo resizing")
227
    if status!=0:
228
      print msg % 'identify'
229
      log.error(msg, identify)
230
      return bufData
231
    try:
232
      if int(ret)<int(GetResizeStr().split('x')[0]):
233
        log.info('image size is smaller than size specified in config.txt;'
234
                 ' retaining original size')
235
        return bufData
236
    except:
237
      log.error('invalid format of image.size in config.txt')
238
      return bufData
239
    log.debug("resizing image %s to size %s" % (im, GetResizeStr()))
240
    cmd = 'convert %s -resize %s %s-conv'%(im, GetResizeStr(), im)
241
    ret = os.system(cmd)
242
    if ret!=0:
243
      print msg % 'convert'
244
      log.error(msg, 'convert')
245
      return bufData
246
    else:
247
      f = open(im + '-conv')
248
      return f.read()
249
250
251
  def writeMetaInfo(self, id, INFO):
252
    #The metadata may be unicode strings, so we need to encode them on write
253
    filePath = os.path.join(flickrfsHome, '.'+id)
254
    f = codecs.open(filePath, 'w', 'utf8')
255
    f.write('# Metadata file : flickrfs - Virtual filesystem for flickr\n')
256
    f.write('# Photo owner: %s NSID: %s\n' % (INFO[7], INFO[8]))
257
    f.write('# Handy link to photo: %s\n'%(INFO[9]))
258
    f.write('# Licences available: \n')
259
    for (k, v) in self.licenses:
260
      f.write('# %s : %s\n' % (k, v))
261
    f.write('[metainfo]\n')
262
    f.write("%s:%s\n"%('title', INFO[4]))
263
    f.write("%s:%s\n"%('description', INFO[3]))
264
    tags = ','.join(INFO[5])
265
    f.write("%s:%s\n"%('tags', tags))
266
    f.write("%s:%s\n"%('license',INFO[6]))
267
    f.close()
268
    f = open(filePath)
269
    f.read()
270
    fileSize = f.tell()
271
    f.close()
272
    return fileSize
273
274
  def __populate_set(self, set_id, curdir):
275
    # Exception handling will be done by background function.
276
    photosInSet = self.transfl.getPhotosFromPhotoset(set_id)
277
    for b,p in photosInSet.iteritems():
278
      info = self.transfl.parseInfoFromPhoto(b,p)
279
      self._mkfileWithMeta(curdir, info)
280
    log.info("set %s populated, photo count %s", curdir, len(photosInSet))
281
282
  def sets_thread(self):
283
    """
284
      The beauty of the FUSE python implementation is that with the 
285
      python interpreter running in foreground, you can have threads
286
    """
287
    print "Sets are being populated in the background."
288
    log.info("started")
289
    self._mkdir("/sets")
290
    for a in self.transfl.getPhotosetList():
291
      title = a.title[0].elementText.replace('/', '_')
292
      log.info("populating set %s", title)
293
      curdir = "/sets/" + title
294
      if title.strip()=='':
295
        curdir = "/sets/" + a['id']
296
      set_id = a['id']
297
      self._mkdir(curdir, id=set_id)
298
      self.__populate_set(set_id, curdir)
299
300
  def _sync_code(self, psetOnline, curdir):
301
    psetLocal = set(map(lambda x: x[0], self.getdir(curdir, False)))
302
    for b in psetOnline:
303
      info = self.transfl.parseInfoFromPhoto(b)
304
      imageTitle = info.get('title','')
305
      if hasattr(b, 'originalformat'):
306
        imageTitle = self.__getImageTitle(imageTitle, 
307
                                        b['id'], b['originalformat'])
308
      else:
309
        imageTitle = self.__getImageTitle(imageTitle, b['id'])
310
      path = "%s/%s"%(curdir, imageTitle)
311
      inode = self.inodeCache.get(path)
312
      # This exception throwing is just for debugging.
313
      if inode == None and self.inodeCache.has_key(path):
314
        e = OSError("Path %s present in inodeCache" % path)
315
        e.errno = ENOENT
316
        raise e
317
      if inode == None: # Image inode not present in the set.
318
        log.debug("new image found: %s", path)
319
        self._mkfileWithMeta(curdir, info)
320
      else:
321
        if inode.mtime != int(info.get('dupdate')):
322
          log.debug("image %s changed", path)
323
          self.inodeCache.pop(path)
324
          if self.inodeCache.has_key(path + ".meta"):
325
            self.inodeCache.pop(path + ".meta")
326
          self._mkfileWithMeta(curdir, info)
327
        psetLocal.discard(imageTitle)
328
    if len(psetLocal)>0:
329
      log.info('%s photos have been deleted online' % len(psetLocal))
330
    for c in psetLocal:
331
      log.info('deleting %s', c)
332
      self.unlink("%s/%s" % (curdir, c), False)
333
334
  def __sync_set_in_background(self, set_id, curdir):
335
    # Exception handling will be done by background function.
336
    log.info("syncing set %s", curdir)
337
    psetOnline = self.transfl.getPhotosFromPhotoset(set_id)
338
    self._sync_code(psetOnline, curdir)
339
    log.info("set %s sync successfully finished", curdir)
340
    
341
  def sync_sets_thread(self):
342
    log.info("started")
343
    setListOnline = self.transfl.getPhotosetList()
344
    setListLocal = self.getdir('/sets', False)
345
    
346
    for a in setListOnline:
347
      title = a.title[0].elementText.replace('/', '_')
348
      if title.strip()=="":
349
        title = a['id']
350
      if (title,0) not in setListLocal: #New set added online
351
        log.info("new set %s found online", title)
352
        self._mkdir('/sets/'+title, a['id'])
353
      else: #Present Online
354
        setListLocal.remove((title,0))
355
    for a in setListLocal: #List of sets present locally, but not online
356
      log.info('set %s no longer online, recursively deleting it', a)
357
      self.rmdir('/sets/'+a[0], online=False, recr=True)
358
        
359
    for a in setListOnline:
360
      title = a.title[0].elementText.replace('/', '_')
361
      curdir = "/sets/" + title
362
      if title.strip()=='':
363
        curdir = "/sets/" + a['id']
364
      set_id = a['id']
365
      self.__sync_set_in_background(set_id, curdir)
366
    log.info('finished')
367
368
  def sync_stream_thread(self):
369
    log.info('started')
370
    psetOnline = self.transfl.getPhotoStream(self.NSID)
371
    self._sync_code(psetOnline, '/stream')
372
    log.info('finished')
373
      
374
  def stream_thread(self):
375
    log.info("started")
376
    print "Populating photostream"
377
    for b in self.transfl.getPhotoStream(self.NSID):
378
      info = self.transfl.parseInfoFromPhoto(b)
379
      self._mkfileWithMeta('/stream', info)
380
    log.info("finished")
381
    print "Photostream population finished."
382
      
383
  def tags_thread(self, path):
384
    ind = string.rindex(path, '/')
385
    tagName = path[ind+1:]
386
    if tagName.strip()=='':
387
      log.error("the tagName '%s' doesn't contain any tags", tagName)
388
      return 
389
    log.info("started for %s", tagName)
390
    sendtagList = ','.join(tagName.split(':'))
391
    if(path.startswith('/tags/personal')):
392
      user_id = self.NSID
393
    else:
394
      user_id = None
395
    for b in self.transfl.getTaggedPhotos(sendtagList, user_id):
396
      info = self.transfl.parseInfoFromPhoto(b)
397
      self._mkfileWithMeta(path, info)
398
399
  def getUnixPerms(self, info):
400
    mode = info.get('mode')
401
    if mode is not None:
402
      return mode
403
    perms = info.get('perms')
404
    if perms is None:
405
      return 0644
406
    if perms is "1": # public
407
      return 0755
408
    elif perms is "2": # friends only. Add 1 to 4 in middle letter.
409
      return 0754
410
    elif perms is "3": # family only. Add 2 to 4 in middle letter.
411
      return 0764
412
    elif perms is "4": # friends and family. Add 1+2 to 4 in middle letter.
413
      return 0774
414
    else:
415
      return 0744 # private
416
417
  def __getImageTitle(self, title, id, format = "jpg"):
418
    temp = title.replace('/', '')
419
#    return "%s_%s.%s" % (temp[:32], id, format)
420
    # Store the photos original name. Thus, when pictures are uploaded
421
    # their names would remain as it is, allowing easy resumption of
422
    # uploading of images, in case some of the photos fail uploading.
423
    return "%s.%s" % (temp, format)
424
425
  def _mkfileWithMeta(self, path, info):
426
    # Don't write the meta information here, because it requires access to
427
    # the full INFO. Only do with the smaller version of information that
428
    # is provided.
429
    if info is None:
430
      return
431
    title = info.get("title", "")
432
    id =    info.get("id", "")
433
    ext =   info.get("format", "jpg")
434
    title = self.__getImageTitle(title, id, ext)
435
436
    # Refactor this section of code, so that it can be called
437
    # from read.
438
    # Figure out a way to retrieve information, which can be 
439
    # used in _mkfile.
440
    mtime = info.get("dupdate")
441
    ctime = info.get("dupload")
442
    perms = self.getUnixPerms(info)
443
    self._mkfile(path+"/"+title, id=id, mode=perms, mtime=mtime, ctime=ctime)
444
    self._mkfile(path+'/.'+title+'.meta', id)
445
446
  def _parsepathid(self, path, id=""):
447
    #Path and Id may be unicode strings, so encode them to utf8 now before
448
    #we use them, otherwise python will throw errors when we combine them
449
    #with regular strings.
450
    path = path.encode('utf8')
451
    if id!=0: id = id.encode('utf8')
452
    parentDir, name = os.path.split(path)
453
    if parentDir=='':
454
      parentDir = '/'
455
    log.debug("parentDir %s", parentDir)
456
    return path, id, parentDir, name
457
458
  def _mkdir(self, path, id="", mtime=None, ctime=None):
459
    path, id, parentDir, name = self._parsepathid(path, id)
460
    log.debug("creating directory %s", path)
461
    self.inodeCache[path] = inodes.DirInode(path, id, mtime=mtime, ctime=ctime)
462
    if path!='/':
463
      pinode = self.getInode(parentDir)
464
      pinode.nlink += 1
465
      self.updateInode(parentDir, pinode)
466
      log.debug("nlink of %s is now %s", parentDir, pinode.nlink)
467
468
  def _mkfile(self, path, id="", mode=None, 
469
              comm_meta="", mtime=None, ctime=None):
470
    path, id, parentDir, name = self._parsepathid(path, id)
471
    log.debug("creating file %s with id %s", path, id)
472
    image_name, extension = os.path.splitext(name)
473
    if not extension:
474
      log.error("can't create file without extension")
475
      return
476
    fInode = inodes.FileInode(path, id, mode=mode, comm_meta=comm_meta,
477
                              mtime=mtime, ctime=ctime)
478
    self.inodeCache[path] = fInode
479
    # Now create the meta info inode if the meta info file exists
480
    # refactoring: create the meta info inode, regardless of the
481
    # existence of datapath.
482
#    path = os.path.join(parentDir, '.' + image_name + '.meta')
483
#    datapath = os.path.join(flickrfsHome, '.'+id)
484
#    if os.path.exists(datapath):
485
#    size = os.path.getsize(datapath)
486
#    self.inodeCache[path] = FileInode(path, id)
487
488
  def getattr(self, path):
489
    # getattr is being called 4-6 times every second for '/'
490
    # Don't log those calls, as they clutter up the log file.
491
    if path != "/":
492
      logattr.debug("getattr: %s", path)
493
    templist = path.split('/')
494
    if path.startswith('/sets/'):
495
      templist[2] = templist[2].split(':')[0]
496
    elif path.startswith('/stream'):
497
      templist[1] = templist[1].split(':')[0]
498
    path = '/'.join(templist)
499
500
    inode=self.getInode(path)
501
    if inode:
502
      #log.debug("inode %s", inode)
503
      statTuple = (inode.mode,inode.ino,inode.dev,inode.nlink,
504
          inode.uid,inode.gid,inode.size,inode.atime,inode.mtime,inode.ctime)
505
      #log.debug("statsTuple %s", statTuple)
506
      return statTuple
507
    else:
508
      e = OSError("No such file"+path)
509
      e.errno = ENOENT
510
      raise e
511
512
  def readlink(self, path):
513
    log.debug("readlink")
514
    return os.readlink(path)
515
  
516
  def getdir(self, path, hidden=True):
517
    logattr.debug("getdir: %s", path)
518
    templist = []
519
    if hidden:
520
      templist = ['.', '..']
521
    for a in self.inodeCache.keys():
522
      ind = a.rindex('/')
523
      if path=='/':
524
        path=""
525
      if path==a[:ind]:
526
        name = a.split('/')[-1]
527
        if name=="":
528
          continue
529
        if hidden and name.startswith('.'):
530
          templist.append(name)
531
        elif not name.startswith('.'):
532
          templist.append(name)
533
    return map(lambda x: (x,0), templist)
534
535
  def unlink(self, path, online=True):
536
    log.debug("unlink %s", path)
537
    if self.inodeCache.has_key(path):
538
      inode = self.inodeCache.pop(path)
539
      # Remove the meta data file as well if it exists
540
      if self.inodeCache.has_key(path + ".meta"):
541
        self.inodeCache.pop(path + ".meta")
542
543
      typesinfo = mimetypes.guess_type(path)
544
      if typesinfo[0] is None or typesinfo[0].count('image')<=0:
545
        log.debug("unlinked non-image file %s", path)
546
        return
547
548
      if path.startswith('/sets/'):
549
        ind = path.rindex('/')
550
        pPath = path[:ind]
551
        pinode = self.getInode(pPath)
552
        if online:
553
          self.transfl.removePhotofromSet(photoId=inode.photoId, 
554
                                          photosetId=pinode.setId)
555
          log.info("photo %s removed from set", path)
556
      del inode
557
    else:
558
      log.error("%s is not a known file", path)
559
      #Dont' raise an exception. Not useful when
560
      #using editors like Vim. They make loads of 
561
      #crap buffer files
562
  
563
  def rmdir(self, path, online=True, recr=False):
564
    log.debug("removing %s", path)
565
    if self.inodeCache.has_key(path):
566
      for a in self.inodeCache.keys():
567
        if a.startswith(path+'/'):
568
          if recr:
569
            self.unlink(a, online)
570
          else:
571
            e = OSError("Directory not empty")
572
            e.errno = ENOTEMPTY
573
            raise e
574
    else:
575
      log.error("%s is not a known directory", path)
576
      e = OSError("No such folder"+path)
577
      e.errno = ENOENT
578
      raise e
579
      
580
    if path=='/sets' or path=='/tags' or path=='/tags/personal' \
581
        or path=='/tags/public' or path=='/stream':
582
      log.debug("attempt to remove framework file %s rejected", path)
583
      e = OSError("removal of folder %s not allowed" % (path))
584
      e.errno = EPERM
585
      raise e
586
587
    ind = path.rindex('/')
588
    pPath = path[:ind]
589
    inode = self.inodeCache.pop(path)
590
    if online and path.startswith('/sets/'):
591
      self.transfl.deleteSet(inode.setId)
592
    del inode
593
    pInode = self.getInode(pPath)
594
    pInode.nlink -= 1
595
    self.updateInode(pPath, pInode)
596
  
597
  def symlink(self, path, path1):
598
    log.debug("symlink")
599
    return os.symlink(path, path1)
600
601
  def rename(self, path, path1):
602
    log.debug("%s %s", path, path1)
603
    #Donot allow Vim to create a file~
604
    #Check for .meta in both paths
605
    if path.count('~')>0 or path1.count('~')>0:
606
      log.debug("vim enablement path entered")
607
      try:
608
        #Get inode, but _dont_ remove from cache
609
        inode = self.getInode(path)
610
        if inode is not None:
611
          self.inodeCache[path1] = inode
612
      except:
613
        log.debug("couldn't find inode for %s", path)
614
      return
615
616
    #Read from path
617
    inode = self.getInode(path)
618
    if inode is None or not hasattr(inode, 'photoId'):
619
      return
620
    fname = os.path.join(flickrfsHome, '.'+inode.photoId)
621
    f = open(fname, 'r')
622
    buf = f.read()
623
    f.close()
624
625
    #Now write to path1
626
    inode = self.getInode(path1)
627
    if inode is None or not hasattr(inode, 'photoId'):
628
      return
629
    fname = os.path.join(flickrfsHome, '.'+inode.photoId)
630
    f = open(fname, 'w')
631
    f.write(buf)
632
    f.close()
633
    inode.size = os.path.getsize(fname)
634
    self.updateInode(path1, inode)
635
    retinfo = self.parse(fname, inode.photoId)
636
    if retinfo.count('Error')>0:
637
      log.error(retinfo)
638
    
639
  def link(self, srcpath, destpath):
640
    log.debug("%s %s", srcpath, destpath)
641
    #Add image from stream to set, w/o retrieving
642
    slist = srcpath.split('/')
643
    sname_file = slist.pop(-1)
644
    dlist = destpath.split('/')
645
    dname_file = dlist.pop(-1)
646
    error = 0
647
    if sname_file=="" or sname_file.startswith('.'):
648
      error = 1
649
    if dname_file=="" or dname_file.startswith('.'):
650
      error = 1
651
    if not destpath.startswith('/sets/'):
652
      error = 1
653
    if error is 1:
654
      log.error("linking is allowed only between 2 image files")
655
      return
656
    sinode = self.getInode(srcpath)
657
    self._mkfile(destpath, id=sinode.id, mode=sinode.mode, 
658
                 comm_meta=sinode.comm_meta, mtime=sinode.mtime, 
659
                 ctime=sinode.ctime)
660
    parentPath = '/'.join(dlist)
661
    pinode = self.getInode(parentPath)
662
    if pinode.setId==0:
663
      try:
664
        pinode.setId = self.transfl.createSet(parentPath, sinode.photoId)
665
        self.updateInode(parentPath, pinode)
666
      except:
667
        e = OSError("Can't create a new set")
668
        e.errno = EIO
669
        raise e
670
    else:
671
      self.transfl.put2Set(pinode.setId, sinode.photoId)
672
673
  
674
  def chmod(self, path, mode):
675
    log.debug("%s %s" % path, mode)
676
    inode = self.getInode(path)
677
    typesinfo = mimetypes.guess_type(path)
678
679
    if inode.comm_meta is None:
680
      log.debug("chmod on directory ignored")
681
      return
682
        
683
    elif typesinfo[0] is None or typesinfo[0].count('image')<=0:
684
      
685
      os.chmod(path, mode)
686
      return
687
688
    elif self.transfl.setPerm(inode.photoId, mode, inode.comm_meta)==True:
689
      inode.mode = mode
690
      self.updateInode(path, inode)
691
      return
692
    
693
  def chown(self, path, user, group):
694
    log.debug("%s:%s %s (ignored)", user, group, path)
695
    
696
  def truncate(self, path, size):
697
    log.debug("%s %s", path, size)
698
    ind = path.rindex('/')
699
    name_file = path[ind+1:]
700
701
    typeinfo = mimetypes.guess_type(path)
702
    if typeinfo[0] is None or typeinfo[0].count('image')<=0:
703
      inode = self.getInode(path)
704
      filePath = os.path.join(flickrfsHome, '.'+inode.photoId)
705
      f = open(filePath, 'w+')
706
      return f.truncate(size)
707
    
708
  def mknod(self, path, mode, dev):
709
    log.debug("%s %s %s ", path, mode, dev)
710
    templist = path.split('/')
711
    name_file = templist[-1]
712
713
    if name_file.startswith('.') and name_file.count('.meta') > 0:
714
      # We need to handle the special case, where some meta files are being
715
      # created through mknod. Creation of meta files is done when adding
716
      # images automatically; and they should not go through mknod system call.
717
      # Editors like Vim, try to generate random swap files when reading
718
      # meta; and this should be *disallowed*.
719
      log.debug("mknod for meta file %s ignored", path)
720
      return
721
722
    if path.startswith('/sets/'):
723
      templist[2] = templist[2].split(':')[0]
724
    elif path.startswith('/stream'):
725
      templist[1] = templist[1].split(':')[0]
726
    path = '/'.join(templist)
727
728
    log.debug("modified file path %s", path)
729
    #Lets guess what kind of a file is this. 
730
    #Is it an image file? or, some other temporary file
731
    #created by the tools you're using. 
732
    typeinfo = mimetypes.guess_type(path)
733
    if typeinfo[0] is None or typeinfo[0].count('image') <= 0:
734
      f = open(os.path.join(flickrfsHome,'.'+name_file), 'w')
735
      f.close()
736
      # TODO(manishrjain): This should not be FileInode, it should rather be
737
      # Inode.
738
      self.inodeCache[path] = inodes.FileInode(path, name_file, mode=mode)
739
    else:
740
      self._mkfile(path, id="NEW", mode=mode)
741
742
  def mkdir(self, path, mode):
743
    log.debug("%s with mode %s", path, mode)
744
    if path.startswith("/tags"):
745
      if path.count('/')==3:   #/tags/personal (or private)/dirname ONLY
746
        self._mkdir(path)
747
        background(self.tags_thread, path)
748
      else:
749
        e = OSError("Not allowed to create directory %s" % path)
750
        e.errno = EACCES
751
        raise e
752
    elif path.startswith("/sets"):
753
      if path.count('/')==2:  #Only allow creation of new set /sets/newset
754
        self._mkdir(path, id=0)
755
          #id=0 means that not yet created online
756
      else:
757
        e = OSError("Not allowed to create directory %s" % path)
758
        e.errno = EACCES
759
        raise e
760
    elif path=='/stream':
761
      self._mkdir(path)
762
      background(timerThread, self.stream_thread, 
763
                 self.sync_stream_thread, stream_sync_int)
764
      
765
    else:
766
      e = OSError("Not allowed to create directory %s" % path)
767
      e.errno = EACCES
768
      raise e
769
      
770
  def utime(self, path, times):
771
    inode = self.getInode(path)
772
    inode.atime = times[0]
773
    inode.mtime = times[1]
774
    self.updateInode(path, inode)
775
    return 0
776
777
  def open(self, path, flags):
778
    log.info("%s with flags %s", path, flags)
779
    ind = path.rindex('/')
780
    name_file = path[ind+1:]
781
    if name_file.startswith('.') and name_file.endswith('.meta'):
782
      self.handleAccessToNonImage(path)
783
      return 0
784
    typesinfo = mimetypes.guess_type(path)
785
    if typesinfo[0] is None or typesinfo[0].count('image')<=0:
786
      log.debug('non-image file found %s', path)
787
      self.handleAccessToNonImage(path)
788
      return 0
789
    
790
    templist = path.split('/')
791
    if path.startswith('/sets/'):
792
      templist[2] = templist[2].split(':')[0]
793
    elif path.startswith('/stream'):
794
      templist[1] = templist[1].split(':')[0]
795
    path = '/'.join(templist)
796
    log.debug("path after modification is %s", path)
797
    
798
    inode = self.getInode(path)
799
    if inode.photoId=="NEW": #Just skip if new (i.e. uploading)
800
      return 0
801
    if self.imgCache.getBuffer(inode.photoId)=="":  
802
      log.debug("retrieving image %s from flickr", inode.photoId)
803
      self.imgCache.setBuffer(inode.photoId,
804
          str(self.transfl.getPhoto(inode.photoId)))
805
      inode.size = self.imgCache.getBufLen(inode.photoId)
806
      log.debug("size of image is %s", inode.size)
807
      self.updateInode(path, inode)
808
    return 0
809
    
810
  def read(self, path, length, offset):
811
    log.debug("%s offset %s length %s", path, offset, length)
812
    ind = path.rindex('/')
813
    name_file = path[ind+1:]
814
    if name_file.startswith('.') and name_file.endswith('.meta'):
815
      # Check if file is not present. If not, retrieve and 
816
      # create the file locally.
817
      buf = self.handleReadNonImage(path, length, offset)
818
      return buf
819
    typesinfo = mimetypes.guess_type(path)
820
    if typesinfo[0] is None or typesinfo[0].count('image')<=0:
821
      return self.handleReadNonImage(path, length, offset)
822
    return self.handleReadImage(path, length, offset)
823
824
  def parse(self, fname, photoId):
825
    cp = ConfigParser.ConfigParser()
826
    log.debug("parsing file %s for photoid %s", fname, photoId)
827
    cp.read(fname)
828
    log.debug("file %s has been read by ConfigParser", fname)
829
    options = cp.options('metainfo')
830
    title=''
831
    desc=''
832
    tags=''
833
    license=''
834
    if 'description' in options:
835
      desc = cp.get('metainfo', 'description')
836
    if 'tags' in options:
837
      tags = cp.get('metainfo', 'tags')
838
    if 'title' in options:
839
      title = cp.get('metainfo', 'title')
840
    if 'license' in options:
841
      license = cp.get('metainfo', 'license')
842
      
843
    log.debug("setting metadata for file %s", fname)
844
    if self.transfl.setMeta(photoId, title, desc)==False:
845
      return "Error:Can't set Meta information"
846
      
847
    log.debug("setting tags for %s", fname)
848
    if self.transfl.setTags(photoId, tags)==False:
849
      log.debug("setting tags for %s failed", fname)
850
      return "Error:Can't set tags"
851
852
    log.debug("setting license for %s", fname)
853
    if self.transfl.setLicense(photoId, license)==False:
854
      return "Error:Can't set license"
855
            
856
 #   except:
857
 #     log.error("Can't parse file:%s:"%(fname,))
858
 #     return "Error:Can't parse"
859
    return 'Success:Updated photo:%s:%s:'%(fname,photoId)
860
861
  ##################################################
862
  # 'handle' Functions for handling read and writes.
863
  ##################################################
864
  def handleAccessToNonImage(self, path):
865
    inode = self.getInode(path)
866
    if inode is None:
867
      log.error("inode %s doesn't exist", path)
868
      e = OSError("No inode found")
869
      e.errno = EIO
870
      raise e
871
    fname = os.path.join(flickrfsHome, '.'+inode.photoId) #ext
872
    # Handle the case when file already exists.
873
    if not os.path.exists(fname) or os.path.getsize(fname) == 0L:
874
      log.info("retrieving meta information for file %s and photo id %s", 
875
               fname, inode.photoId)
876
      INFO = self.transfl.getPhotoInfo(inode.photoId)
877
      size = self.writeMetaInfo(inode.photoId, INFO)
878
      log.info("information has been written for photo id %s", inode.photoId)
879
      inode.size = size
880
      self.updateInode(path, inode)
881
      time.sleep(1) # Enough time for OS to call for getattr again.
882
    return inode
883
884
  def handleReadNonImage(self, path, length, offset):
885
    inode = self.handleAccessToNonImage(path)
886
    f = open(os.path.join(flickrfsHome, '.'+inode.photoId), 'r')
887
    f.seek(offset)
888
    return f.read(length)
889
  
890
  def handleReadImage(self, path, length, offset):
891
    inode = self.getInode(path)
892
    if inode is None:
893
      log.error("inode %s doesn't exist", path)
894
      e = OSError("No inode found")
895
      e.errno = EIO
896
      raise e
897
    if self.imgCache.getBufLen(inode.photoId) is 0:  
898
      log.debug("retrieving image %s from flickr", inode.photoId)
899
      buf = retryFlickrOp(False, self.transfl.getPhoto,
900
                          inode.photoId)
901
      if len(buf) == 0:
902
        log.error("can't retrieve image %s", inode.photoId)
903
        e = OSError("Unable to retrieve image.")
904
        e.errno = EIO
905
        raise e
906
      self.imgCache.setBuffer(inode.photoId, buf)
907
      inode.size = self.imgCache.getBufLen(inode.photoId)
908
    temp =  self.imgCache.getBuffer(inode.photoId, offset, offset+length)
909
    if len(temp) < length:
910
      self.imgCache.popBuffer(inode.photoId)
911
    self.updateInode(path, inode)
912
    return temp
913
914
  def handleWriteToNonImage(self, path, buf, off):
915
    inode = self.handleAccessToNonImage(path)
916
    fname = os.path.join(flickrfsHome, '.'+inode.photoId) #ext
917
    log.debug("writing to %s", fname)
918
    f = open(fname, 'r+')
919
    f.seek(off)
920
    f.write(buf)
921
    f.close()
922
    if len(buf)<4096:
923
      inode.size = os.path.getsize(fname)
924
      retinfo = self.parse(fname, inode.photoId)
925
      if retinfo.count('Error')>0:
926
        e = OSError(retinfo.split(':')[1])
927
        e.errno = EIO
928
        raise e
929
      self.updateInode(path, inode)
930
    return len(buf)
931
932
  def handleUploadingImage(self, path, inode, taglist):
933
    tags = [ '"%s"'%(a,) for a in taglist]
934
    tags.append('flickrfs')
935
    taglist = ' '.join(tags)
936
    log.info('uploading %s with len %s', 
937
             path, self.imgCache.getBufLen(inode.photoId))
938
    id = None
939
    bufData = self.imgCache.getBuffer(inode.photoId)
940
    bufData = self.imageResize(bufData)
941
    id = retryFlickrOp(False, self.transfl.uploadfile,
942
                       path, taglist, bufData, inode.mode)
943
    if id is None:
944
      log.error("unable to upload file %s", inode.photoId)
945
      e = OSError("Unable to upload file.")
946
      e.errno = EIO
947
      raise e
948
    self.imgCache.popBuffer(inode.photoId)
949
    inode.photoId = id
950
    self.updateInode(path, inode)
951
    return inode
952
953
  def handleWriteToBuffer(self, path, buf):
954
    inode = self.getInode(path)
955
    if inode is None:
956
      log.error("inode %s doesn't exist", path)
957
      e = OSError("No inode found")
958
      e.errno = EIO
959
      raise e
960
    self.imgCache.addBuffer(inode.photoId, buf)
961
    return inode
962
963
  def handleWriteAddToSet(self, parentPath, pinode, inode):
964
    #Create set if it doesn't exist online (i.e. if id=0)
965
    if pinode.setId is 0:
966
      # Retry creation of set if unsuccessful.
967
      pinode.setId = retryFlickrOp(False, self.transfl.createSet,
968
                                   parentPath, inode.photoId)
969
      # If the set is created, then return.
970
      if pinode.setId is not None:
971
        self.updateInode(parentPath, pinode)
972
        return
973
      else:
974
        log.error("unable to create set %s", parentPath)
975
        e = OSError("Unable to create set.")
976
        e.errno = EIO
977
        raise e
978
    else:
979
      # If the operation put2Set doesn't throw exception, that means
980
      # that the picture has been successfully added to set.
981
      # Return in that case, retry otherwise.
982
      retryFlickrOp(True, self.transfl.put2Set,
983
                    pinode.setId, inode.photoId)
984
      return
985
986
  #############################
987
  # End of 'handle' Functions.
988
  #############################
989
990
  def write(self, path, buf, off):
991
    log.debug("write to %s at offset %s", path, off)
992
    ind = path.rindex('/')
993
    name_file = path[ind+1:]
994
    if name_file.startswith('.') and name_file.count('.meta')>0:
995
      return self.handleWriteToNonImage(path, buf, off)
996
    typesinfo = mimetypes.guess_type(path)
997
    if typesinfo[0] is None or typesinfo[0].count('image')<=0:
998
      return self.handleWriteToNonImage(path, buf, off)
999
    templist = path.split('/')
1000
    inode = None
1001
    if path.startswith('/tags'):
1002
      e = OSError("Copying to tags not allowed")
1003
      e.errno = EIO
1004
      raise e
1005
    if path.startswith('/stream'):
1006
      tags = templist[1].split(':')
1007
      templist[1] = tags.pop(0)
1008
      path = '/'.join(templist)
1009
      inode = self.handleWriteToBuffer(path, buf)
1010
      if len(buf) < 4096:
1011
        self.handleUploadingImage(path, inode, tags)
1012
    elif path.startswith('/sets/'):
1013
      setnTags = templist[2].split(':')
1014
      setName = setnTags.pop(0)
1015
      templist[2] = setName
1016
      path = '/'.join(templist)
1017
      inode = self.handleWriteToBuffer(path, buf)
1018
      if len(buf) < 4096:
1019
        templist.pop(-1)
1020
        parentPath = '/'.join(templist)
1021
        pinode = self.getInode(parentPath)
1022
        inode = self.handleUploadingImage(path, inode, setnTags)
1023
        self.handleWriteAddToSet(parentPath, pinode, inode)
1024
    log.debug("done write to %s at offset %s", path, off)
1025
    if len(buf)<4096:
1026
      templist = path.split('/')
1027
      templist.pop(-1)
1028
      parentPath = '/'.join(templist)
1029
      try:
1030
        self.inodeCache.pop(path)
1031
      except:
1032
        pass
1033
      INFO = self.transfl.getPhotoInfo(inode.photoId)
1034
      info = self.transfl.parseInfoFromFullInfo(inode.photoId, INFO)
1035
      self._mkfileWithMeta(parentPath, info)
1036
      self.writeMetaInfo(inode.photoId, INFO)
1037
    return len(buf)
1038
1039
  def getInode(self, path):
1040
    if self.inodeCache.has_key(path):
1041
      #log.debug("got cached inode for %s", path)
1042
      return self.inodeCache[path]
1043
    else:
1044
      #log.debug("%s is not in inode cache", path)
1045
      return None
1046
1047
  def updateInode(self, path, inode):
1048
    self.inodeCache[path] = inode
1049
1050
  def release(self, path, flags):
1051
    log.debug("%s with flags %s ignored", path, flags)
1052
    return 0
1053
  
1054
  def statfs(self):
1055
    """
1056
  Should return a tuple with the following elements in respective order:
1057
  
1058
  F_BSIZE - Preferred file system block size. (int)
1059
  F_FRSIZE - Fundamental file system block size. (int)
1060
  F_BLOCKS - Total number of blocks in the filesystem. (long)
1061
  F_BFREE - Total number of free blocks. (long)
1062
  F_BAVAIL - Free blocks available to non-super user. (long)
1063
  F_FILES - Total number of file nodes. (long)
1064
  F_FFREE - Total number of free file nodes. (long)
1065
  F_FAVAIL - Free nodes available to non-super user. (long)
1066
  F_FLAG - Flags. System dependent: see statvfs() man page. (int)
1067
  F_NAMEMAX - Maximum file name length. (int)
1068
  Feel free to set any of the above values to 0, which tells
1069
  the kernel that the info is not available.
1070
    """
1071
    block_size = 1024
1072
    blocks = 0L
1073
    blocks_free = 0L
1074
    files = 0L
1075
    files_free = 0L
1076
    namelen = 255
1077
    # statfs is being called repeatedly at least once a second.
1078
    # The bandwidth information doesn't change that often, so
1079
    # save upon communication with flickr servers to retrieve this
1080
    # information. Only retrieve it once in a while.
1081
    if self.statfsCounter >= 500 or self.statfsCounter is -1:
1082
      (self.max, self.used) = self.transfl.getBandwidthInfo()
1083
      self.statfsCounter = 0
1084
      log.info('retrieved bandwidth info: max %s used %s', 
1085
               self.max, self.used)
1086
    self.statfsCounter = self.statfsCounter + 1
1087
1088
    if self.max is not None:
1089
      blocks = long(self.max)/block_size
1090
      blocks_used = long(self.used)/block_size
1091
      blocks_free = blocks - blocks_used
1092
      blocks_available = blocks_free
1093
      return (block_size, blocks, blocks_free, blocks_available, 
1094
              files, files_free, namelen)
1095
1096
  def fsync(self, path, isfsyncfile):
1097
    log.debug("%s,  isfsyncfile=%s", path, isfsyncfile)
1098
    return 0
1099
1100
1101
if __name__ == '__main__':
1102
  try:
1103
    server = Flickrfs()
1104
    server.multithreaded = 1;
1105
    server.main()
1106
  except KeyError:
1107
    log.error('got key error; exiting...')
1108
    sys.exit(0)
(-)flickrfs-1.3.9.orig/flickrfs/inodes.py (+134 lines)
Line 0 Link Here
1
#===============================================================================
2
#  flickrfs - Virtual Filesystem for Flickr
3
#  Copyright (c) 2005,2006 Manish Rai Jain  <manishrjain@gmail.com>
4
#
5
#  This program can be distributed under the terms of the GNU GPL version 2, or 
6
#  its later versions. 
7
#
8
# DISCLAIMER: The API Key and Shared Secret are provided by the author in 
9
# the hope that it will prevent unnecessary trouble to the end-user. The 
10
# author will not be liable for any misuse of this API Key/Shared Secret 
11
# through this application/derived apps/any 3rd party apps using this key. 
12
#===============================================================================
13
14
__author__ =  "Manish Rai Jain (manishrjain@gmail.com)"
15
__license__ = "GPLv2 (details at http://www.gnu.org/licenses/licenses.html#GPL)"
16
17
import os, sys, time
18
from stat import *
19
import cPickle
20
21
DEFAULTBLOCKSIZE = 4*1024 # 4 KB
22
23
class Inode(object):
24
  """Common base class for all file system objects
25
  """
26
  def __init__(self, path=None, id='', mode=None, 
27
               size=0L, mtime=None, ctime=None):
28
    self.nlink = 1
29
    self.size = size 
30
    self.id = id
31
    self.mode = mode
32
    self.ino = long(time.time())
33
    self.dev = 409089L
34
    self.uid = int(os.getuid())
35
    self.gid = int(os.getgid())
36
    now = int(time.time())
37
    self.atime = now
38
    if mtime is None:
39
      self.mtime = now
40
    else:
41
      self.mtime = int(mtime)
42
    if ctime is None:
43
      self.ctime = now
44
    else:
45
      self.ctime = int(ctime)
46
    self.blocksize = DEFAULTBLOCKSIZE
47
48
class DirInode(Inode):
49
  def __init__(self, path=None, id="", mode=None, mtime=None, ctime=None):
50
    if mode is None: mode = 0755
51
    super(DirInode, self).__init__(path, id, mode, 0L, mtime, ctime)
52
    self.mode = S_IFDIR | self.mode
53
    self.nlink += 1
54
    self.dirfile = ""
55
    self.setId = self.id
56
57
58
class FileInode(Inode):
59
  def __init__(self, path=None, id="", mode=None, comm_meta="", 
60
               size=0L, mtime=None, ctime=None):
61
    if mode is None: mode = 0644
62
    super(FileInode, self).__init__(path, id, mode, size, mtime, ctime)
63
    self.mode = S_IFREG | self.mode
64
    self.photoId = self.id
65
    self.comm_meta = comm_meta
66
67
68
class ImageCache:
69
  def __init__(self):
70
    self.bufDict = {}
71
72
  def setBuffer(self, id, buf):
73
    self.bufDict[id] = buf
74
75
  def addBuffer(self, id, inc):
76
    buf = self.getBuffer(id)
77
    self.setBuffer(id, buf+inc)
78
79
  def getBuffer(self, id, start=0, end=0):
80
    if end == 0:
81
      return self.bufDict.get(id, "")[start:]
82
    else:
83
      return self.bufDict.get(id, "")[start:end]
84
85
  def getBufLen(self, id):
86
    return long(len(self.bufDict.get(id, "")))
87
88
  def popBuffer(self, id):
89
    if id in self.bufDict:
90
      return self.bufDict.pop(id)
91
92
93
class InodeCache(dict):
94
  def __init__(self, dbPath):
95
    dict.__init__(self)
96
    try:
97
      import bsddb
98
      # If bsddb is available, utilize that package
99
      # and store the inodes in database.
100
      self.db = bsddb.btopen(dbPath, flag='c')
101
    except:
102
      # Otherwise, store the inodes in memory.
103
      self.db = {}
104
    # Keep the keys in memory.
105
    self.keysCache = set()
106
  
107
  def __getitem__(self, key, d=None):
108
    # key k may be unicode, so convert it to
109
    # a normal string first.
110
    if not self.has_key(key):
111
      return d
112
    valObjStr = self.db.get(str(key))
113
    return cPickle.loads(valObjStr)
114
115
  def __setitem__(self, key, value):
116
    self.keysCache.add(key)
117
    self.db[str(key)] = cPickle.dumps(value)
118
  
119
  def get(self, k, d=None):
120
    return self.__getitem__(k, d)
121
122
  def keys(self):
123
    return list(self.keysCache)
124
  
125
  def pop(self, k, *args):
126
    # key k may be unicode, so convert it to
127
    # a normal string first.
128
    valObjStr = self.db.pop(str(k), *args)
129
    self.keysCache.discard(k)
130
    if valObjStr != None:
131
      return cPickle.loads(valObjStr)
132
133
  def has_key(self, k):
134
    return k in self.keysCache
(-)flickrfs-1.3.9.orig/flickrfs/transactions.py (+443 lines)
Line 0 Link Here
1
#===============================================================================
2
#  flickrfs - Virtual Filesystem for Flickr
3
#  Copyright (c) 2005,2006 Manish Rai Jain  <manishrjain@gmail.com>
4
#
5
#  This program can be distributed under the terms of the GNU GPL version 2, or 
6
#  its later versions. 
7
#
8
# DISCLAIMER: The API Key and Shared Secret are provided by the author in 
9
# the hope that it will prevent unnecessary trouble to the end-user. The 
10
# author will not be liable for any misuse of this API Key/Shared Secret 
11
# through this application/derived apps/any 3rd party apps using this key. 
12
#===============================================================================
13
14
__author__ =  "Manish Rai Jain (manishrjain@gmail.com)"
15
__license__ = "GPLv2 (details at http://www.gnu.org/licenses/licenses.html#GPL)"
16
17
from flickrapi import FlickrAPI
18
from traceback import format_exc
19
import urllib2
20
import sys
21
import string
22
import os
23
import time
24
import logging
25
log = logging.getLogger('flickrfs.trans')
26
27
# flickr auth information
28
flickrAPIKey = "f8aa9917a9ae5e44a87cae657924f42d"  # API key
29
flickrSecret = "3fbf7144be7eca28"  # shared "secret"
30
31
# Utility functions
32
def kwdict(**kw): return kw
33
34
#Transactions with flickr, wraps FlickrAPI 
35
# calls in Flickfs-specialized functions.
36
class TransFlickr: 
37
38
  extras = "original_format,date_upload,last_update"
39
40
  def __init__(self, browserName):
41
    self.fapi = FlickrAPI(flickrAPIKey, flickrSecret)
42
    self.user_id = ""
43
    # proceed with auth
44
    # TODO use auth.checkToken function if available, 
45
    # and wait after opening browser.
46
    print "Authorizing with flickr..."
47
    log.info("authorizing with flickr...")
48
    try:
49
      self.authtoken = self.fapi.getToken(browser=browserName)
50
    except:
51
      print ("Can't retrieve token from browser %s" % browserName)
52
      print ("\tIf you're behind a proxy server,"
53
             " first set http_proxy environment variable.")
54
      print "\tPlease close all your browser windows, and try again"
55
      log.error(format_exc())
56
      log.error("can't retrieve token from browser %s", browserName)
57
      sys.exit(-1)
58
    if self.authtoken == None:
59
      print "Unable to authorize (reason unknown)"
60
      log.error('not able to authorize; exiting')
61
      sys.exit(-1)
62
        #Add some authorization checks here(?)
63
    print "Authorization complete."
64
    log.info('authorization complete')
65
    
66
  def uploadfile(self, filepath, taglist, bufData, mode):
67
    #Set public 4(always), 1(public). Public overwrites f&f.
68
    public = mode&1
69
    #Set friends and family 4(always), 2(family), 1(friends).
70
    friends = mode>>3 & 1
71
    family = mode>>4 & 1
72
      #E.g. 745 - 4:No f&f, but 5:public
73
      #E.g. 754 - 5:friends, but not public
74
      #E.g. 774 - 7:f&f, but not public
75
76
    log.info("uploading file %s", filepath)
77
    log.info("  data length: %s", len(bufData))
78
    log.info("  taglist: %s", taglist)
79
    log.info("  permissions: family %s, friends %s, public %s",
80
             family, friends, public)
81
    filename = os.path.splitext(os.path.basename(filepath))[0]
82
    rsp = self.fapi.upload(filename=filepath, jpegData=bufData,
83
          title=filename,
84
          tags=taglist,
85
          is_public=public and "1" or "0",
86
          is_friend=friends and "1" or "0",
87
          is_family=family and "1" or "0")
88
89
    if rsp is None:
90
      log.error("response None from attempt to write file %s", filepath)
91
      log.error("will attempt recovery...")
92
      recent_rsp = None
93
      trytimes = 2
94
      while(trytimes):
95
        log.info("sleeping for 3 seconds...")
96
        time.sleep(3)
97
        trytimes -= 1
98
        # Keep on trying to retrieve the recently uploaded photo, till we
99
        # actually get the information, or the function throws an exception.
100
        while(recent_rsp is None or not recent_rsp):
101
          recent_rsp = self.fapi.photos_recentlyUpdated(
102
              auth_token=self.authtoken, min_date='1', per_page='1')
103
        
104
        pic = recent_rsp.photos[0].photo[0]
105
        log.info('we are looking for %s', filename)
106
        log.info('most recently updated pic is %s', pic['title'])
107
        if filename == pic['title']:
108
          id = pic['id']
109
          log.info("file %s uploaded with photoid %s", filepath, id)
110
          return id
111
      log.error("giving up; upload of %s appears to have failed", filepath)
112
      return None
113
    else:
114
      id = rsp.photoid[0].elementText
115
      log.info("file %s uploaded with photoid %s", filepath, id)
116
      return id
117
118
  def put2Set(self, set_id, photo_id):
119
    log.info("uploading photo %s to set id %s", photo_id, set_id)
120
    rsp = self.fapi.photosets_addPhoto(auth_token=self.authtoken, 
121
                                       photoset_id=set_id, photo_id=photo_id)
122
    if rsp:
123
      log.info("photo uploaded to set")
124
    else:
125
      log.error(rsp.errormsg)
126
  
127
  def createSet(self, path, photo_id):
128
    log.info("creating set %s with primary photo %s", path, photo_id)
129
    path, title = os.path.split(path)
130
    rsp = self.fapi.photosets_create(auth_token=self.authtoken, 
131
                                     title=title, primary_photo_id=photo_id)
132
    if rsp:
133
      log.info("created set %s", title)
134
      return rsp.photoset[0]['id']
135
    else:
136
      log.error(rsp.errormsg)
137
  
138
  def deleteSet(self, set_id):
139
    log.info("deleting set %s", set_id)
140
    if str(set_id)=="0":
141
      log.info("ignoring attempt to delete set wtih set_id 0 (a locally "
142
        "created set that has not yet acquired an id via uploading")
143
      return
144
    rsp = self.fapi.photosets_delete(auth_token=self.authtoken, 
145
                                     photoset_id=set_id)
146
    if rsp:
147
      log.info("deleted set %s", set_id)
148
    else:
149
      log.error(rsp.errormsg)
150
  
151
  def getPhotoInfo(self, photoId):
152
    log.debug("id: %s", photoId)
153
    rsp = self.fapi.photos_getInfo(auth_token=self.authtoken, photo_id=photoId)
154
    if not rsp:
155
      log.error("can't retrieve information about photo %s; got error %s",
156
                photoId, rsp.errormsg)
157
      return None
158
    #XXX: should see if there's some other 'format' option we can fall back to.
159
    try: format = rsp.photo[0]['originalformat']
160
    except KeyError: format = 'jpg'
161
    perm_public = rsp.photo[0].visibility[0]['ispublic']
162
    perm_family = rsp.photo[0].visibility[0]['isfamily']
163
    perm_friend = rsp.photo[0].visibility[0]['isfriend']
164
    if perm_public == '1':
165
      mode = 0755
166
    else:
167
      b_cnt = 4
168
      if perm_family == '1':
169
        b_cnt += 2
170
      if perm_friend == '1':
171
        b_cnt += 1
172
      mode = "07" + str(b_cnt) + "4"
173
      mode = int(mode)
174
      
175
    if hasattr(rsp.photo[0],'permissions'):
176
      permcomment = rsp.photo[0].permissions[0]['permcomment']
177
      permaddmeta = rsp.photo[0].permissions[0]['permaddmeta']
178
    else:
179
      permcomment = permaddmeta = [None]
180
      
181
    commMeta = '%s%s' % (permcomment,permaddmeta) # Required for chmod.
182
    desc = rsp.photo[0].description[0].elementText
183
    title = rsp.photo[0].title[0].elementText
184
    if hasattr(rsp.photo[0].tags[0], "tag"):
185
      taglist = [ a.elementText for a in rsp.photo[0].tags[0].tag ]
186
    else:
187
      taglist = []
188
    license = rsp.photo[0]['license']
189
    owner = rsp.photo[0].owner[0]['username']
190
    ownerNSID = rsp.photo[0].owner[0]['nsid']
191
    url = rsp.photo[0].urls[0].url[0].elementText
192
    posted = rsp.photo[0].dates[0]['posted']
193
    lastupdate = rsp.photo[0].dates[0]['lastupdate']
194
    return (format, mode, commMeta, desc, title, taglist, 
195
            license, owner, ownerNSID, url, int(posted), int(lastupdate))
196
197
  def setPerm(self, photoId, mode, comm_meta="33"):
198
    log.debug("id: %s, mode: %s, comm_meta=%s", photoId, mode, comm_meta)
199
    public = mode&1 #Set public 4(always), 1(public). Public overwrites f&f
200
    #Set friends and family 4(always), 2(family), 1(friends) 
201
    friends = mode>>3 & 1
202
    family = mode>>4 & 1
203
    if len(comm_meta)<2: 
204
      # This wd patch string index out of range bug, caused 
205
      # because some photos may not have comm_meta value set.
206
      comm_meta="33"
207
    rsp = self.fapi.photos_setPerms(auth_token=self.authtoken,
208
                                    is_public=str(public),
209
                                    is_friend=str(friends), 
210
                                    is_family=str(family), 
211
                                    perm_comment=comm_meta[0],
212
                                    perm_addmeta=comm_meta[1], 
213
                                    photo_id=photoId)
214
    if not rsp:
215
      log.error("couldn't set permission for photo %s; got error %s",
216
                photoId, rsp.errormsg)
217
      return False
218
    log.info("permissions have been set for photo %s", photoId)
219
    return True
220
221
  def setTags(self, photoId, tags):
222
    log.debug("id: %s, tags: %s", photoId, tags)
223
    templist = [ '"%s"'%(a,) for a in string.split(tags, ',')] + ['flickrfs']
224
    tagstring = ' '.join(templist)
225
    rsp = self.fapi.photos_setTags(auth_token=self.authtoken, 
226
                                   photo_id=photoId, tags=tagstring)
227
    if not rsp:
228
      log.error("couldn't set tags for %s; got error %s",
229
                photoId, rsp.errormsg)
230
      return False
231
    return True
232
  
233
  def setMeta(self, photoId, title, desc):
234
    log.debug("id: %s, title: %s, desc: %s", photoId, title, desc)
235
    rsp = self.fapi.photos_setMeta(auth_token=self.authtoken, 
236
                                   photo_id=photoId, title=title, 
237
                                   description=desc)
238
    if not rsp:
239
      log.error("couldn't set meta info for photo %s; got error",
240
                photoId, rsp.errormsg)
241
      return False
242
    return True
243
244
  def getLicenses(self):
245
    log.debug("started")
246
    rsp = self.fapi.photos_licenses_getInfo()
247
    if not rsp:
248
      log.error("couldn't retrieve licenses; got error %s", rsp.errormsg)
249
      return None
250
    licenseDict = {}
251
    for l in rsp.licenses[0].license:
252
      licenseDict[l['id']] = l['name']
253
    keys = licenseDict.keys()
254
    keys.sort()
255
    sortedLicenseList = []
256
    for k in keys:
257
      # Add tuple of license key, and license value.
258
      sortedLicenseList.append((k, licenseDict[k]))
259
    return sortedLicenseList
260
    
261
  def setLicense(self, photoId, license):
262
    log.debug("id: %s, license: %s", photoId, license)
263
    rsp = self.fapi.photos_licenses_setLicense(auth_token=self.authtoken, 
264
                                               photo_id=photoId, 
265
                                               license_id=license)
266
    if not rsp:
267
      log.error("couldn't set license info for photo %s; got error %s",
268
                photoId, rsp.errormsg)
269
      return False
270
    return True
271
272
  def getPhoto(self, photoId):
273
    log.debug("id: %s", photoId)
274
    rsp = self.fapi.photos_getSizes(auth_token=self.authtoken, 
275
                                    photo_id=photoId)
276
    if not rsp:
277
      log.error("error while trying to retrieve size information"
278
                " for photo %s", photoId)
279
      return None
280
    buf = ""
281
    for a in rsp.sizes[0].size:
282
      if a['label']=='Original':
283
        try:
284
          f = urllib2.urlopen(a['source'])
285
          buf = f.read()
286
        except:
287
          log.error("exception in getPhoto")
288
          log.error(format_exc())
289
          return ""
290
    if not buf:
291
      f = urllib2.urlopen(rsp.sizes[0].size[-1]['source'])
292
      buf = f.read()
293
    return buf
294
295
  def removePhotofromSet(self, photoId, photosetId):
296
    log.debug("id: %s, setid: %s", photoId, photosetId)
297
    rsp = self.fapi.photosets_removePhoto(auth_token=self.authtoken, 
298
                                          photo_id=photoId, 
299
                                          photoset_id=photosetId)
300
    if rsp:
301
      log.info("photo %s removed from set %s", photoId, photosetId)
302
    else:
303
      log.error(rsp.errormsg)
304
      
305
    
306
  def getBandwidthInfo(self):
307
    log.debug("retrieving bandwidth information")
308
    rsp = self.fapi.people_getUploadStatus(auth_token=self.authtoken)
309
    if not rsp:
310
      log.error("can't retrieve bandwidth information; got error %s",
311
        rsp.errormsg)
312
      return (None,None)
313
    bw = rsp.user[0].bandwidth[0]
314
    log.debug("max bandwidth: %s, bandwidth used: %s", bw['max'], bw['used'])
315
    return (bw['max'], bw['used'])
316
317
  def getUserId(self):
318
    log.debug("entered")
319
    rsp = self.fapi.auth_checkToken(api_key=flickrAPIKey, 
320
                                    auth_token=self.authtoken)
321
    if not rsp:
322
      log.error("unable to get userid; got error %s", rsp.errormsg)
323
      return None
324
    usr = rsp.auth[0].user[0]
325
    log.info("got NSID %s", usr['nsid'])
326
    #Set self.user_id to this value
327
    self.user_id = usr['nsid']
328
    return usr['nsid']
329
330
  def getPhotosetList(self):
331
    log.debug("entered")
332
    if self.user_id is "":
333
      self.getUserId() #This will set the value of self.user_id
334
    rsp = self.fapi.photosets_getList(auth_token=self.authtoken, 
335
                                      user_id=self.user_id)
336
    if not rsp:
337
      log.error("error getting photoset list; got error %s", rsp.errormsg)
338
      return []
339
    if not hasattr(rsp.photosets[0], "photoset"):
340
      log.info("no sets found for userid %s", self.user_id)
341
      return []
342
    else:
343
      log.info("%s sets found for userid %s",
344
          len(rsp.photosets[0].photoset), self.user_id)
345
    return rsp.photosets[0].photoset
346
347
  def parseInfoFromPhoto(self, photo, perms=None):
348
    info = {}
349
    info['id'] = photo['id']
350
    info['title'] = photo['title'].replace('/', '_')
351
    # Some pics don't contain originalformat attribute, so set it to jpg by default.
352
    try:
353
      info['format'] = photo['originalformat']
354
    except KeyError:
355
      info['format'] = 'jpg'
356
357
    try:
358
      info['dupload'] = photo['dateupload']
359
    except KeyError:
360
      info['dupload'] = '0'
361
362
    try:
363
      info['dupdate'] = photo['lastupdate']
364
    except KeyError:
365
      info['dupdate'] = '0'
366
    
367
    info['perms'] = perms
368
    return info
369
370
  def parseInfoFromFullInfo(self, id, fullInfo):
371
    info = {}
372
    info['id'] = id
373
    info['title'] = fullInfo[4]
374
    info['format'] = fullInfo[0]
375
    info['dupload'] = fullInfo[10]
376
    info['dupdate'] = fullInfo[11]
377
    info['mode'] = fullInfo[1]
378
    return info
379
380
  def getPhotosFromPhotoset(self, photoset_id):
381
    log.debug("set id: %s", photoset_id)
382
    photosPermsMap = {}
383
    # I'm not utilizing the value part of this dictionary. Its arbitrarily
384
    # set to i.
385
    for i in range(0,3):
386
      page = 1
387
      while True:
388
        rsp = self.fapi.photosets_getPhotos(auth_token=self.authtoken,
389
                                            photoset_id=photoset_id, 
390
                                            extras=self.extras, 
391
                                            page=str(page),
392
                                            privacy_filter=str(i))
393
        if not rsp:
394
          break
395
        if not hasattr(rsp.photoset[0], 'photo'):
396
          log.error("photoset %s doesn't have attribute photo", rsp.photoset[0]['id'])
397
          break
398
        for p in rsp.photoset[0].photo:
399
          photosPermsMap[p] = str(i)
400
        page += 1
401
        if page > int(rsp.photoset[0]['pages']): break
402
      if photosPermsMap: break
403
    return photosPermsMap
404
            
405
  def getPhotoStream(self, user_id):
406
    log.debug("userid: %s", user_id)
407
    retList = []
408
    pageNo = 1
409
    maxPage = 1
410
    while pageNo<=maxPage:
411
      log.info("retreiving page number %s of %s", pageNo, maxPage) 
412
      rsp = self.fapi.photos_search(auth_token=self.authtoken, 
413
                                    user_id=user_id, per_page="500", 
414
                                    page=str(pageNo), extras=self.extras)
415
      if not rsp:
416
        log.error("can't retrive photos from your stream; got error %s",
417
            rsp.errormsg)
418
        return retList
419
      if not hasattr(rsp.photos[0], 'photo'):
420
        log.error("photos.search response doesn't have attribute photos; "
421
            "returning list acquired so far")
422
        return retList
423
      for a in rsp.photos[0].photo:
424
        retList.append(a)
425
      maxPage = int(rsp.photos[0]['pages'])
426
      pageNo = pageNo + 1
427
    return retList
428
 
429
  def getTaggedPhotos(self, tags, user_id=None):
430
    log.debug("tags: %s user_id: %s", tags, user_id)
431
    kw = kwdict(auth_token=self.authtoken, tags=tags, tag_mode="all", 
432
                extras=self.extras, per_page="500")
433
    if user_id is not None: 
434
      kw = kwdict(user_id=user_id, **kw)
435
    rsp = self.fapi.photos_search(**kw)
436
    log.debug("search for photos with tags %s has been"
437
              " successfully finished" % tags)
438
    if not rsp:
439
      log.error("couldn't search for the photos; got error %s", rsp.errormsg)
440
      return
441
    if not hasattr(rsp.photos[0], 'photo'):
442
      return []
443
    return rsp.photos[0].photo
(-)flickrfs-1.3.9.orig/flickrfs.py (-1084 lines)
Lines 1-1084 Link Here
1
#===============================================================================
2
#  flickrfs - Virtual Filesystem for Flickr
3
#  Copyright (c) 2005,2006 Manish Rai Jain  <manishrjain@gmail.com>
4
#
5
#  This program can be distributed under the terms of the GNU GPL version 2, or 
6
#  its later versions. 
7
#
8
# DISCLAIMER: The API Key and Shared Secret are provided by the author in 
9
# the hope that it will prevent unnecessary trouble to the end-user. The 
10
# author will not be liable for any misuse of this API Key/Shared Secret 
11
# through this application/derived apps/any 3rd party apps using this key. 
12
#===============================================================================
13
14
__author__ =  "Manish Rai Jain (manishrjain@gmail.com)"
15
__license__ = "GPLv2 (details at http://www.gnu.org/licenses/licenses.html#GPL)"
16
17
import thread, string, ConfigParser, mimetypes, codecs
18
import time, logging, logging.handlers, os, sys
19
from glob import glob
20
from errno import *
21
from traceback import format_exc
22
from fuse import Fuse
23
import threading
24
import random, commands
25
from urllib2 import URLError
26
from transactions import TransFlickr
27
import inodes
28
29
#Some global definitions and functions
30
NUMRETRIES = 3
31
32
#Set up the .flickfs directory.
33
homedir = os.getenv('HOME')
34
flickrfsHome = os.path.join(homedir, '.flickrfs')
35
dbPath = os.path.join(flickrfsHome, '.inode.bdb')
36
37
if not os.path.exists(flickrfsHome):
38
  os.mkdir(os.path.join(flickrfsHome))
39
else:
40
  # Remove previous metadata files from ~/.flickrfs
41
  for a in glob(os.path.join(flickrfsHome, '.*')):
42
    os.remove(os.path.join(flickrfsHome, a))
43
  try:
44
    os.remove(dbPath)
45
  except:
46
    pass
47
 
48
# Added by Varun Hiremath
49
if not os.path.exists(flickrfsHome + "/config.txt"):
50
  fconfig = open(flickrfsHome+"/config.txt",'w')
51
  fconfig.write("[configuration]\n")
52
  fconfig.write("browser:/usr/bin/firefox\n")
53
  fconfig.write("image.size:\n")
54
  fconfig.write("sets.sync.int:300\n")
55
  fconfig.write("stream.sync.int:300\n")
56
  fconfig.close()
57
58
# Set up logging
59
log = logging.getLogger('flickrfs')
60
loghdlr = logging.handlers.RotatingFileHandler(
61
                             os.path.join(flickrfsHome,'log'), "a", 5242880, 3)
62
logfmt = logging.Formatter("%(asctime)s %(levelname)-10s %(message)s", "%x %X")
63
loghdlr.setFormatter(logfmt)
64
log.addHandler(loghdlr)
65
log.setLevel(logging.DEBUG)
66
67
cp = ConfigParser.ConfigParser()
68
cp.read(flickrfsHome + '/config.txt')
69
_resizeStr = ""
70
sets_sync_int = 600.0
71
stream_sync_int = 600.0
72
try:
73
  _resizeStr = cp.get('configuration', 'image.size')
74
except:
75
  print 'No default size of image found. Will upload original size of images.'
76
try:
77
  sets_sync_int = float(cp.get('configuration', 'sets.sync.int'))
78
except:
79
  pass
80
try:
81
  stream_sync_int = float(cp.get('configuration', 'stream.sync.int'))
82
except:
83
  pass
84
try:
85
  browserName = cp.get('configuration', 'browser')
86
except:
87
  pass
88
89
# Retrive the resize string.
90
def GetResizeStr():
91
  return _resizeStr
92
93
#Utility functions.
94
def _log_exception_wrapper(func, *args, **kw):
95
  """Call 'func' with args and kws and log any exception it throws.
96
  """
97
  for i in range(0, NUMRETRIES):
98
    log.debug("Retry attempt %s for func %s" % (i, func))
99
    try:
100
      func(*args, **kw)
101
      return
102
    except:
103
      log.error("Exception in function %s" % func)
104
      log.error(format_exc())
105
106
def background(func, *args, **kw):
107
    """Run 'func' as a thread, logging any exceptions it throws.
108
109
    To run
110
111
      somefunc(arg1, arg2='value')
112
113
    as a thread, do:
114
115
      background(somefunc, arg1, arg2='value')
116
117
    Any exceptions thrown are logged as errors, and the traceback is logged.
118
    """
119
    thread.start_new_thread(_log_exception_wrapper, (func,)+args, kw)
120
121
def timerThread(func, func1, interval):
122
  '''Execute func now, followed by func1 every interval seconds
123
  '''
124
  t = threading.Timer(0.0, func)
125
  try:
126
    t.run()
127
  except: pass
128
  while(interval):
129
    t = threading.Timer(interval, func1)
130
    try:
131
      t.run()
132
    except: pass
133
134
def retryFlickrOp(isNone, func, *args):
135
  # This function helps in retrying the flickr transactions, in case they fail.
136
  result = None
137
  for i in range(0, NUMRETRIES):
138
    log.debug("Retry attempt %s for func %s" % (i, func))
139
    try:
140
      result = func(*args)
141
      if result is None:
142
        if isNone:
143
          return result
144
        else:
145
          continue
146
      else:
147
        return result
148
    except URLError, detail:
149
      log.error("Failure in function %s with error: %s" % (func, detail))
150
  # We've utilized all our attempts, send out the result whatever it is.
151
  return result
152
153
class Flickrfs(Fuse):
154
155
  def __init__(self, *args, **kw):
156
  
157
    Fuse.__init__(self, *args, **kw)
158
    log.info("flickrfs.py:Flickrfs:mountpoint: %s" % repr(self.mountpoint))
159
    log.info("flickrfs.py:Flickrfs:unnamed mount options: %s" % self.optlist)
160
    log.info("flickrfs.py:Flickrfs:named mount options: %s" % self.optdict)
161
    
162
    self.inodeCache = inodes.InodeCache(dbPath) # Inodes need to be stored.
163
    self.imgCache = inodes.ImageCache()
164
    self.NSID = ""
165
    self.transfl = TransFlickr(log, browserName)
166
167
    # Set some variables to be utilized by statfs function.
168
    self.statfsCounter = -1
169
    self.max = 0L
170
    self.used = 0L
171
172
    self.NSID = self.transfl.getUserId()
173
    if self.NSID is None:
174
      log.error("Initialization:Can't retrieve user information")
175
      sys.exit(-1)
176
177
    log.info('Getting list of licenses available')
178
    self.licenses = self.transfl.getLicenses()
179
    if self.licenses is None:
180
      log.error("Initialization:Can't retrieve license information")
181
      sys.exit(-1)
182
183
    # do stuff to set up your filesystem here, if you want
184
    self._mkdir("/")
185
    self._mkdir("/tags")
186
    self._mkdir("/tags/personal")
187
    self._mkdir("/tags/public")
188
    background(timerThread, self.sets_thread, 
189
               self.sync_sets_thread, sets_sync_int) #sync every 2 minutes
190
191
192
  def imageResize(self, bufData):
193
    # If no resizing information is present, then return the buffer directly.
194
    if GetResizeStr() == "":
195
      return bufData
196
197
    # Else go ahead and do the conversion.
198
    im = '/tmp/flickrfs-' + str(int(random.random()*1000000000))
199
    f = open(im, 'w')
200
    f.write(bufData)
201
    f.close()
202
    cmd = 'identify -format "%%w" %s'%(im,)
203
    status,ret = commands.getstatusoutput(cmd)
204
    if status!=0:
205
      print "identify command not found. Install Imagemagick"
206
      log.error("identify command not found. Install Imagemagick")
207
      return bufData
208
    try:
209
      if int(ret)<int(GetResizeStr().split('x')[0]):
210
        log.info('Image size is smaller than specified in config.txt.'
211
                 ' Taking original size')
212
        return bufData
213
    except:
214
      log.error('Invalid format of image.size in config.txt')
215
      return bufData
216
    log.debug("Resizing image %s to size %s" % (im, GetResizeStr()))
217
    cmd = 'convert %s -resize %s %s-conv'%(im, GetResizeStr(), im)
218
    ret = os.system(cmd)
219
    if ret!=0:
220
      print "convert Command not found. Install Imagemagick"
221
      log.error("convert Command not found. Install Imagemagick")  
222
      return bufData
223
    else:
224
      f = open(im + '-conv')
225
      return f.read()
226
227
228
  def writeMetaInfo(self, id, INFO):
229
    #The metadata may be unicode strings, so we need to encode them on write
230
    filePath = os.path.join(flickrfsHome, '.'+id)
231
    f = codecs.open(filePath, 'w', 'utf8')
232
    f.write('# Metadata file : flickrfs - Virtual filesystem for flickr\n')
233
    f.write('# Photo owner: %s NSID: %s\n' % (INFO[7], INFO[8]))
234
    f.write('# Handy link to photo: %s\n'%(INFO[9]))
235
    f.write('# Licences available: \n')
236
    for (k, v) in self.licenses:
237
      f.write('# %s : %s\n' % (k, v))
238
    f.write('[metainfo]\n')
239
    f.write("%s:%s\n"%('title', INFO[4]))
240
    f.write("%s:%s\n"%('description', INFO[3]))
241
    tags = ','.join(INFO[5])
242
    f.write("%s:%s\n"%('tags', tags))
243
    f.write("%s:%s\n"%('license',INFO[6]))
244
    f.close()
245
    f = open(filePath)
246
    f.read()
247
    fileSize = f.tell()
248
    f.close()
249
    return fileSize
250
251
  def __populate_set(self, set_id, curdir):
252
    # Exception handling will be done by background function.
253
    photosInSet = self.transfl.getPhotosFromPhotoset(set_id)
254
    for b,p in photosInSet.iteritems():
255
      info = self.transfl.parseInfoFromPhoto(b,p)
256
      self._mkfileWithMeta(curdir, info)
257
    log.info("Set %s populated." % curdir)
258
259
  def sets_thread(self):
260
    """
261
      The beauty of the FUSE python implementation is that with the 
262
      python interpreter running in foreground, you can have threads
263
    """
264
    print "Sets are being populated in the background."
265
    log.info("sets_thread: started")
266
    self._mkdir("/sets")
267
    for a in self.transfl.getPhotosetList():
268
      title = a.title[0].elementText.replace('/', '_')
269
      log.info("Populating set %s." % title)
270
      curdir = "/sets/" + title
271
      if title.strip()=='':
272
        curdir = "/sets/" + a['id']
273
      set_id = a['id']
274
      self._mkdir(curdir, id=set_id)
275
      background(self.__populate_set, set_id, curdir)
276
277
  def _sync_code(self, psetOnline, curdir):
278
    psetLocal = set(map(lambda x: x[0], self.getdir(curdir, False)))
279
    for b in psetOnline:
280
      info = self.transfl.parseInfoFromPhoto(b)
281
      imageTitle = info.get('title','')
282
      imageTitle = self.__getImageTitle(imageTitle, 
283
                                        b['id'], b['originalformat'])
284
      path = "%s/%s"%(curdir, imageTitle)
285
      inode = self.inodeCache.get(path)
286
      # This exception throwing is just for debugging.
287
      if inode == None and self.inodeCache.has_key(path):
288
        e = OSError("Path %s present in inodeCache" % path)
289
        e.errno = ENOENT
290
        raise e
291
      if inode == None: # Image inode not present in the set.
292
        log.debug("New image found: %s"%(path))
293
        self._mkfileWithMeta(curdir, info)
294
      else:
295
        if inode.mtime != int(info.get('dupdate')):
296
          log.debug("Image %s changed"%(path))
297
          self.inodeCache.pop(path)
298
          if self.inodeCache.has_key(path + ".meta"):
299
            self.inodeCache.pop(path + ".meta")
300
          self._mkfileWithMeta(curdir, info)
301
        psetLocal.discard(imageTitle)
302
    if len(psetLocal)>0:
303
      log.info('%s photos have been deleted online' % len(psetLocal))
304
    for c in psetLocal:
305
      log.info('deleting:%s' % c)
306
      self.unlink("%s/%s" % (curdir, c), False)
307
308
  def __sync_set_in_background(self, set_id, curdir):
309
    # Exception handling will be done by background function.
310
    log.info("Syncing set %s" % curdir)
311
    psetOnline = self.transfl.getPhotosFromPhotoset(set_id)
312
    self._sync_code(psetOnline, curdir)
313
    log.info("Set %s sync successfully finished." % curdir)
314
    
315
  def sync_sets_thread(self):
316
    log.info("sync_sets_thread: started")
317
    setListOnline = self.transfl.getPhotosetList()
318
    setListLocal = self.getdir('/sets', False)
319
    
320
    for a in setListOnline:
321
      title = a.title[0].elementText.replace('/', '_')
322
      if title.strip()=="":
323
        title = a['id']
324
      if (title,0) not in setListLocal: #New set added online
325
        log.info("%s set has been added online."%(title,))
326
        self._mkdir('/sets/'+title, a['id'])
327
      else: #Present Online
328
        setListLocal.remove((title,0))
329
    for a in setListLocal: #List of sets present locally, but not online
330
      log.info('Recursively deleting set %s'%(a,))
331
      self.rmdir('/sets/'+a[0], online=False, recr=True)
332
        
333
    for a in setListOnline:
334
      title = a.title[0].elementText.replace('/', '_')
335
      curdir = "/sets/" + title
336
      if title.strip()=='':
337
        curdir = "/sets/" + a['id']
338
      set_id = a['id']
339
      background(self.__sync_set_in_background, set_id, curdir)
340
    log.info('sync_sets_thread finished')
341
342
  def sync_stream_thread(self):
343
    log.info('sync_stream_thread started')
344
    psetOnline = self.transfl.getPhotoStream(self.NSID)
345
    self._sync_code(psetOnline, '/stream')
346
    log.info('sync_stream_thread finished')
347
      
348
  def stream_thread(self):
349
    log.info("stream_thread started")
350
    print "Populating photostream"
351
    for b in self.transfl.getPhotoStream(self.NSID):
352
      info = self.transfl.parseInfoFromPhoto(b)
353
      self._mkfileWithMeta('/stream', info)
354
    log.info("stream_thread finished")
355
    print "Photostream population finished."
356
      
357
  def tags_thread(self, path):
358
    ind = string.rindex(path, '/')
359
    tagName = path[ind+1:]
360
    if tagName.strip()=='':
361
      log.error("The tagName:%s: doesn't contain any tags"%(tagName))
362
      return 
363
    log.info("tags_thread:" + tagName + ":started")
364
    sendtagList = ','.join(tagName.split(':'))
365
    if(path.startswith('/tags/personal')):
366
      user_id = self.NSID
367
    else:
368
      user_id = None
369
    for b in self.transfl.getTaggedPhotos(sendtagList, user_id):
370
      info = self.transfl.parseInfoFromPhoto(b)
371
      self._mkfileWithMeta(path, info)
372
373
  def getUnixPerms(self, info):
374
    mode = info.get('mode')
375
    if mode is not None:
376
      return mode
377
    perms = info.get('perms')
378
    if perms is None:
379
      return 0644
380
    if perms is "1": # public
381
      return 0755
382
    elif perms is "2": # friends only. Add 1 to 4 in middle letter.
383
      return 0754
384
    elif perms is "3": # family only. Add 2 to 4 in middle letter.
385
      return 0764
386
    elif perms is "4": # friends and family. Add 1+2 to 4 in middle letter.
387
      return 0774
388
    else:
389
      return 0744 # private
390
391
  def __getImageTitle(self, title, id, format = "jpg"):
392
    temp = title.replace('/', '')
393
#    return "%s_%s.%s" % (temp[:32], id, format)
394
    # Store the photos original name. Thus, when pictures are uploaded
395
    # their names would remain as it is, allowing easy resumption of
396
    # uploading of images, in case some of the photos fail uploading.
397
    return "%s.%s" % (temp, format)
398
399
  def _mkfileWithMeta(self, path, info):
400
    # Don't write the meta information here, because it requires access to
401
    # the full INFO. Only do with the smaller version of information that
402
    # is provided.
403
    if info is None:
404
      return
405
    title = info.get("title", "")
406
    id =    info.get("id", "")
407
    ext =   info.get("format", "jpg")
408
    title = self.__getImageTitle(title, id, ext)
409
410
    # Refactor this section of code, so that it can be called
411
    # from read.
412
    # Figure out a way to retrieve information, which can be 
413
    # used in _mkfile.
414
    mtime = info.get("dupdate")
415
    ctime = info.get("dupload")
416
    perms = self.getUnixPerms(info)
417
    self._mkfile(path+"/"+title, id=id, mode=perms, mtime=mtime, ctime=ctime)
418
    self._mkfile(path+'/.'+title+'.meta', id)
419
420
  def _parsepathid(self, path, id=""):
421
    #Path and Id may be unicode strings, so encode them to utf8 now before
422
    #we use them, otherwise python will throw errors when we combine them
423
    #with regular strings.
424
    path = path.encode('utf8')
425
    if id!=0: id = id.encode('utf8')
426
    parentDir, name = os.path.split(path)
427
    if parentDir=='':
428
      parentDir = '/'
429
    log.debug("parentDir:" + parentDir + ":")
430
    return path, id, parentDir, name
431
432
  def _mkdir(self, path, id="", mtime=None, ctime=None):
433
    path, id, parentDir, name = self._parsepathid(path, id)
434
    log.debug("Creating directory:" + path)
435
    self.inodeCache[path] = inodes.DirInode(path, id, mtime=mtime, ctime=ctime)
436
    if path!='/':
437
      pinode = self.getInode(parentDir)
438
      pinode.nlink += 1
439
      self.updateInode(parentDir, pinode)
440
      log.debug("nlink of %s is now %s" % (parentDir, pinode.nlink))
441
442
  def _mkfile(self, path, id="", mode=None, 
443
              comm_meta="", mtime=None, ctime=None):
444
    path, id, parentDir, name = self._parsepathid(path, id)
445
    log.debug("Creating file:" + path + ":with id:" + id)
446
    image_name, extension = os.path.splitext(name)
447
    if not extension:
448
      log.error("Can't create file without extension")
449
      return
450
    fInode = inodes.FileInode(path, id, mode=mode, comm_meta=comm_meta,
451
                              mtime=mtime, ctime=ctime)
452
    self.inodeCache[path] = fInode
453
    # Now create the meta info inode if the meta info file exists
454
    # refactoring: create the meta info inode, regardless of the
455
    # existence of datapath.
456
#    path = os.path.join(parentDir, '.' + image_name + '.meta')
457
#    datapath = os.path.join(flickrfsHome, '.'+id)
458
#    if os.path.exists(datapath):
459
#    size = os.path.getsize(datapath)
460
#    self.inodeCache[path] = FileInode(path, id)
461
462
  def getattr(self, path):
463
    # getattr is being called 4-6 times every second for '/'
464
    # Don't log those calls, as they clutter up the log file.
465
    if path != "/":
466
      log.debug("getattr:" + path + ":")
467
    templist = path.split('/')
468
    if path.startswith('/sets/'):
469
      templist[2] = templist[2].split(':')[0]
470
    elif path.startswith('/stream'):
471
      templist[1] = templist[1].split(':')[0]
472
    path = '/'.join(templist)
473
474
    inode=self.getInode(path)
475
    if inode:
476
      #log.debug("inode "+str(inode))
477
      statTuple = (inode.mode,inode.ino,inode.dev,inode.nlink,
478
          inode.uid,inode.gid,inode.size,inode.atime,inode.mtime,inode.ctime)
479
      #log.debug("statsTuple "+str(statTuple))
480
      return statTuple
481
    else:
482
      e = OSError("No such file"+path)
483
      e.errno = ENOENT
484
      raise e
485
486
  def readlink(self, path):
487
    log.debug("readlink")
488
    return os.readlink(path)
489
  
490
  def getdir(self, path, hidden=True):
491
    log.debug("getdir:" + path)
492
    templist = []
493
    if hidden:
494
      templist = ['.', '..']
495
    for a in self.inodeCache.keys():
496
      ind = a.rindex('/')
497
      if path=='/':
498
        path=""
499
      if path==a[:ind]:
500
        name = a.split('/')[-1]
501
        if name=="":
502
          continue
503
        if hidden and name.startswith('.'):
504
          templist.append(name)
505
        elif not name.startswith('.'):
506
          templist.append(name)
507
    return map(lambda x: (x,0), templist)
508
509
  def unlink(self, path, online=True):
510
    log.debug("unlink:%s:" % (path))
511
    if self.inodeCache.has_key(path):
512
      inode = self.inodeCache.pop(path)
513
      # Remove the meta data file as well if it exists
514
      if self.inodeCache.has_key(path + ".meta"):
515
        self.inodeCache.pop(path + ".meta")
516
517
      typesinfo = mimetypes.guess_type(path)
518
      if typesinfo[0] is None or typesinfo[0].count('image')<=0:
519
        log.debug("unlinked a non-image file:%s:"%(path,))
520
        return
521
522
      if path.startswith('/sets/'):
523
        ind = path.rindex('/')
524
        pPath = path[:ind]
525
        pinode = self.getInode(pPath)
526
        if online:
527
          self.transfl.removePhotofromSet(photoId=inode.photoId, 
528
                                          photosetId=pinode.setId)
529
          log.info("Photo %s removed from set"%(path,))
530
      del inode
531
    else:
532
      log.error("Can't find what you want to remove")
533
      #Dont' raise an exception. Not useful when
534
      #using editors like Vim. They make loads of 
535
      #crap buffer files
536
  
537
  def rmdir(self, path, online=True, recr=False):
538
    log.debug("rmdir:%s:"%(path))
539
    if self.inodeCache.has_key(path):
540
      for a in self.inodeCache.keys():
541
        if a.startswith(path+'/'):
542
          if recr:
543
            self.unlink(a, online)
544
          else:
545
            e = OSError("Directory not empty")
546
            e.errno = ENOTEMPTY
547
            raise e
548
    else:
549
      log.error("Can't find the directory you want to remove")
550
      e = OSError("No such folder"+path)
551
      e.errno = ENOENT
552
      raise e
553
      
554
    if path=='/sets' or path=='/tags' or path=='/tags/personal' \
555
        or path=='/tags/public' or path=='/stream':
556
      log.debug("rmdir on the framework! I refuse to do anything! <Stubborn>")
557
      e = OSError("Removal of folder %s not allowed" % (path))
558
      e.errno = EPERM
559
      raise e
560
561
    ind = path.rindex('/')
562
    pPath = path[:ind]
563
    inode = self.inodeCache.pop(path)
564
    if online and path.startswith('/sets/'):
565
      self.transfl.deleteSet(inode.setId)
566
    del inode
567
    pInode = self.getInode(pPath)
568
    pInode.nlink -= 1
569
    self.updateInode(pPath, pInode)
570
  
571
  def symlink(self, path, path1):
572
    log.debug("symlink")
573
    return os.symlink(path, path1)
574
575
  def rename(self, path, path1):
576
    log.debug("rename:path:%s:to path1:%s:"%(path,path1))
577
    #Donot allow Vim to create a file~
578
    #Check for .meta in both paths
579
    if path.count('~')>0 or path1.count('~')>0:
580
      log.debug("This seems Vim working")
581
      try:
582
        #Get inode, but _dont_ remove from cache
583
        inode = self.getInode(path)
584
        if inode is not None:
585
          self.inodeCache[path1] = inode
586
      except:
587
        log.debug("Couldn't find inode for:%s:"%(path,))
588
      return
589
590
    #Read from path
591
    inode = self.getInode(path)
592
    if inode is None or not hasattr(inode, 'photoId'):
593
      return
594
    fname = os.path.join(flickrfsHome, '.'+inode.photoId)
595
    f = open(fname, 'r')
596
    buf = f.read()
597
    f.close()
598
599
    #Now write to path1
600
    inode = self.getInode(path1)
601
    if inode is None or not hasattr(inode, 'photoId'):
602
      return
603
    fname = os.path.join(flickrfsHome, '.'+inode.photoId)
604
    f = open(fname, 'w')
605
    f.write(buf)
606
    f.close()
607
    inode.size = os.path.getsize(fname)
608
    self.updateInode(path1, inode)
609
    retinfo = self.parse(fname, inode.photoId)
610
    if retinfo.count('Error')>0:
611
      log.error(retinfo)
612
    
613
  def link(self, srcpath, destpath):
614
    log.debug("link: %s:%s"%(srcpath, destpath))
615
    #Add image from stream to set, w/o retrieving
616
    slist = srcpath.split('/')
617
    sname_file = slist.pop(-1)
618
    dlist = destpath.split('/')
619
    dname_file = dlist.pop(-1)
620
    error = 0
621
    if sname_file=="" or sname_file.startswith('.'):
622
      error = 1
623
    if dname_file=="" or dname_file.startswith('.'):
624
      error = 1
625
    if not destpath.startswith('/sets/'):
626
      error = 1
627
    if error is 1:
628
      log.error("Linking is allowed only between 2 image files")
629
      return
630
    sinode = self.getInode(srcpath)
631
    self._mkfile(destpath, id=sinode.id, mode=sinode.mode, 
632
                 comm_meta=sinode.comm_meta, mtime=sinode.mtime, 
633
                 ctime=sinode.ctime)
634
    parentPath = '/'.join(dlist)
635
    pinode = self.getInode(parentPath)
636
    if pinode.setId==0:
637
      try:
638
        pinode.setId = self.transfl.createSet(parentPath, sinode.photoId)
639
        self.updateInode(parentPath, pinode)
640
      except:
641
        e = OSError("Can't create a new set")
642
        e.errno = EIO
643
        raise e
644
    else:
645
      self.transfl.put2Set(pinode.setId, sinode.photoId)
646
647
  
648
  def chmod(self, path, mode):
649
    log.debug("chmod:%s" % path)
650
    inode = self.getInode(path)
651
    typesinfo = mimetypes.guess_type(path)
652
653
    if inode.comm_meta is None:
654
      log.debug("chmod on directory? No use la!")
655
      return
656
        
657
    elif typesinfo[0] is None or typesinfo[0].count('image')<=0:
658
      
659
      os.chmod(path, mode)
660
      return
661
662
    elif self.transfl.setPerm(inode.photoId, mode, inode.comm_meta)==True:
663
      inode.mode = mode
664
      self.updateInode(path, inode)
665
      return
666
    
667
  def chown(self, path, user, group):
668
    log.debug("chown. Are you of any use in flickrfs?")
669
    
670
  def truncate(self, path, size):
671
    log.debug("truncate?? Okay okay! I accept your usage:%s:%s"%(path,size))
672
    ind = path.rindex('/')
673
    name_file = path[ind+1:]
674
675
    typeinfo = mimetypes.guess_type(path)
676
    if typeinfo[0] is None or typeinfo[0].count('image')<=0:
677
      inode = self.getInode(path)
678
      filePath = os.path.join(flickrfsHome, '.'+inode.photoId)
679
      f = open(filePath, 'w+')
680
      return f.truncate(size)
681
    
682
  def mknod(self, path, mode, dev):
683
    """ Python has no os.mknod, so we can only do some things """
684
    log.debug("mknod? OK! Had a close encounter!!:%s:"%(path,))
685
    templist = path.split('/')
686
    name_file = templist[-1]
687
688
    if name_file.startswith('.') and name_file.count('.meta') > 0:
689
      # We need to handle the special case, where some meta files are being
690
      # created through mknod. Creation of meta files is done when adding
691
      # images automatically; and they should not go through mknod system call.
692
      # Editors like Vim, try to generate random swap files when reading
693
      # meta; and this should be *disallowed*.
694
      log.debug("mknod for meta file %s? No use!" % path)
695
      return
696
697
    if path.startswith('/sets/'):
698
      templist[2] = templist[2].split(':')[0]
699
    elif path.startswith('/stream'):
700
      templist[1] = templist[1].split(':')[0]
701
    path = '/'.join(templist)
702
703
    log.debug("mknod: Modified file path %s" % path)
704
    #Lets guess what kind of a file is this. 
705
    #Is it an image file? or, some other temporary file
706
    #created by the tools you're using. 
707
    typeinfo = mimetypes.guess_type(path)
708
    if typeinfo[0] is None or typeinfo[0].count('image') <= 0:
709
      f = open(os.path.join(flickrfsHome,'.'+name_file), 'w')
710
      f.close()
711
      # TODO(manishrjain): This should not be FileInode, it should rather be
712
      # Inode.
713
      self.inodeCache[path] = inodes.FileInode(path, name_file, mode=mode)
714
    else:
715
      self._mkfile(path, id="NEW", mode=mode)
716
717
  def mkdir(self, path, mode):
718
    log.debug("mkdir:" + path + ":")
719
    if path.startswith("/tags"):
720
      if path.count('/')==3:   #/tags/personal (or private)/dirname ONLY
721
        self._mkdir(path)
722
        background(self.tags_thread, path)
723
      else:
724
        e = OSError("Not allowed to create directory %s" % path)
725
        e.errno = EACCES
726
        raise e
727
    elif path.startswith("/sets"):
728
      if path.count('/')==2:  #Only allow creation of new set /sets/newset
729
        self._mkdir(path, id=0)
730
          #id=0 means that not yet created online
731
      else:
732
        e = OSError("Not allowed to create directory %s" % path)
733
        e.errno = EACCES
734
        raise e
735
    elif path=='/stream':
736
      self._mkdir(path)
737
      background(timerThread, self.stream_thread, 
738
                 self.sync_stream_thread, stream_sync_int)
739
      
740
    else:
741
      e = OSError("Not allowed to create directory %s" % path)
742
      e.errno = EACCES
743
      raise e
744
      
745
  def utime(self, path, times):
746
    inode = self.getInode(path)
747
    inode.atime = times[0]
748
    inode.mtime = times[1]
749
    self.updateInode(path, inode)
750
    return 0
751
752
  def open(self, path, flags):
753
    log.info("open: " + path)
754
    ind = path.rindex('/')
755
    name_file = path[ind+1:]
756
    if name_file.startswith('.') and name_file.endswith('.meta'):
757
      self.handleAccessToNonImage(path)
758
      return 0
759
    typesinfo = mimetypes.guess_type(path)
760
    if typesinfo[0] is None or typesinfo[0].count('image')<=0:
761
      log.debug('open: non-image file found %s' % path)
762
      self.handleAccessToNonImage(path)
763
      return 0
764
    
765
    templist = path.split('/')
766
    if path.startswith('/sets/'):
767
      templist[2] = templist[2].split(':')[0]
768
    elif path.startswith('/stream'):
769
      templist[1] = templist[1].split(':')[0]
770
    path = '/'.join(templist)
771
    log.debug("open:After modifying:%s:" % (path))
772
    
773
    inode = self.getInode(path)
774
    if inode.photoId=="NEW": #Just skip if new (i.e. uploading)
775
      return 0
776
    if self.imgCache.getBuffer(inode.photoId)=="":  
777
      log.debug("Retrieving image from flickr: " + inode.photoId)
778
      self.imgCache.setBuffer(inode.photoId,
779
          str(self.transfl.getPhoto(inode.photoId)))
780
      inode.size = self.imgCache.getBufLen(inode.photoId)
781
      log.debug("Size of image: " + str(inode.size))
782
      self.updateInode(path, inode)
783
    return 0
784
    
785
  def read(self, path, length, offset):
786
    log.debug("read:%s:offset:%s:length:%s:" % (path,offset,length))
787
    ind = path.rindex('/')
788
    name_file = path[ind+1:]
789
    if name_file.startswith('.') and name_file.endswith('.meta'):
790
      # Check if file is not present. If not, retrieve and 
791
      # create the file locally.
792
      buf = self.handleReadNonImage(path, length, offset)
793
      return buf
794
    typesinfo = mimetypes.guess_type(path)
795
    if typesinfo[0] is None or typesinfo[0].count('image')<=0:
796
      return self.handleReadNonImage(path, length, offset)
797
    return self.handleReadImage(path, length, offset)
798
799
  def parse(self, fname, photoId):
800
    cp = ConfigParser.ConfigParser()
801
    log.debug("Parsing file %s" % fname)
802
    cp.read(fname)
803
    log.debug("Read file %s for parsing." % fname)
804
    options = cp.options('metainfo')
805
    title=''
806
    desc=''
807
    tags=''
808
    license=''
809
    if 'description' in options:
810
      desc = cp.get('metainfo', 'description')
811
    if 'tags' in options:
812
      tags = cp.get('metainfo', 'tags')
813
    if 'title' in options:
814
      title = cp.get('metainfo', 'title')
815
    if 'license' in options:
816
      license = cp.get('metainfo', 'license')
817
      
818
    log.debug("Setting metadata for file %s" % fname)
819
    if self.transfl.setMeta(photoId, title, desc)==False:
820
      return "Error:Can't set Meta information"
821
      
822
    log.debug("Setting tags:%s:"%(fname,))
823
    if self.transfl.setTags(photoId, tags)==False:
824
      log.debug("Setting tags FAILED : %s" % fname)
825
      return "Error:Can't set tags"
826
827
    log.debug("Setting license:%s:"%(fname,))
828
    if self.transfl.setLicense(photoId, license)==False:
829
      return "Error:Can't set license"
830
            
831
 #   except:
832
 #     log.error("Can't parse file:%s:"%(fname,))
833
 #     return "Error:Can't parse"
834
    return 'Success:Updated photo:%s:%s:'%(fname,photoId)
835
836
  ##################################################
837
  # 'handle' Functions for handling read and writes.
838
  ##################################################
839
  def handleAccessToNonImage(self, path):
840
    inode = self.getInode(path)
841
    if inode is None:
842
      log.error("inode doesn't exist:%s:"%(path,))
843
      e = OSError("No inode found")
844
      e.errno = EIO
845
      raise e
846
    fname = os.path.join(flickrfsHome, '.'+inode.photoId) #ext
847
    # Handle the case when file already exists.
848
    if not os.path.exists(fname) or os.path.getsize(fname) == 0L:
849
      log.info("Retrieving meta information for file %s and photo id %s" % 
850
               (fname, inode.photoId))
851
      INFO = self.transfl.getPhotoInfo(inode.photoId)
852
      size = self.writeMetaInfo(inode.photoId, INFO)
853
      log.info("Information has been written for photo id %s" % inode.photoId)
854
      inode.size = size
855
      self.updateInode(path, inode)
856
      time.sleep(1) # Enough time for OS to call for getattr again.
857
    return inode
858
859
  def handleReadNonImage(self, path, length, offset):
860
    inode = self.handleAccessToNonImage(path)
861
    f = open(os.path.join(flickrfsHome, '.'+inode.photoId), 'r')
862
    f.seek(offset)
863
    return f.read(length)
864
  
865
  def handleReadImage(self, path, length, offset):
866
    inode = self.getInode(path)
867
    if inode is None:
868
      log.error("inode doesn't exist:%s:"%(path,))
869
      e = OSError("No inode found")
870
      e.errno = EIO
871
      raise e
872
    if self.imgCache.getBufLen(inode.photoId) is 0:  
873
      log.debug("Retrieving image from flickr: " + inode.photoId)
874
      buf = retryFlickrOp(False, self.transfl.getPhoto,
875
                          inode.photoId)
876
      if len(buf) == 0:
877
        log.error("Can't retrieve image %s"%(inode.photoId,))
878
        e = OSError("Unable to retrieve image.")
879
        e.errno = EIO
880
        raise e
881
      self.imgCache.setBuffer(inode.photoId, buf)
882
      inode.size = self.imgCache.getBufLen(inode.photoId)
883
    temp =  self.imgCache.getBuffer(inode.photoId, offset, offset+length)
884
    if len(temp) < length:
885
      self.imgCache.popBuffer(inode.photoId)
886
    self.updateInode(path, inode)
887
    return temp
888
889
  def handleWriteToNonImage(self, path, buf, off):
890
    inode = self.handleAccessToNonImage(path)
891
    fname = os.path.join(flickrfsHome, '.'+inode.photoId) #ext
892
    log.debug("Writing to :%s:"%(fname,))
893
    f = open(fname, 'r+')
894
    f.seek(off)
895
    f.write(buf)
896
    f.close()
897
    if len(buf)<4096:
898
      inode.size = os.path.getsize(fname)
899
      retinfo = self.parse(fname, inode.photoId)
900
      if retinfo.count('Error')>0:
901
        e = OSError(retinfo.split(':')[1])
902
        e.errno = EIO
903
        raise e
904
      self.updateInode(path, inode)
905
    return len(buf)
906
907
  def handleUploadingImage(self, path, inode, taglist):
908
    tags = [ '"%s"'%(a,) for a in taglist]
909
    tags.append('flickrfs')
910
    taglist = ' '.join(tags)
911
    log.info('uploading %s with len %s' % 
912
             (path, self.imgCache.getBufLen(inode.photoId)))
913
    id = None
914
    bufData = self.imgCache.getBuffer(inode.photoId)
915
    bufData = self.imageResize(bufData)
916
    id = retryFlickrOp(False, self.transfl.uploadfile,
917
                       path, taglist, bufData, inode.mode)
918
    if id is None:
919
      log.error("unable to upload file:%s:"%(inode.photoId,))
920
      e = OSError("Unable to upload file.")
921
      e.errno = EIO
922
      raise e
923
    self.imgCache.popBuffer(inode.photoId)
924
    inode.photoId = id
925
    self.updateInode(path, inode)
926
    return inode
927
928
  def handleWriteToBuffer(self, path, buf):
929
    inode = self.getInode(path)
930
    if inode is None:
931
      log.error("inode doesn't exist:%s:"%(path,))
932
      e = OSError("No inode found")
933
      e.errno = EIO
934
      raise e
935
    self.imgCache.addBuffer(inode.photoId, buf)
936
    return inode
937
938
  def handleWriteAddToSet(self, parentPath, pinode, inode):
939
    #Create set if it doesn't exist online (i.e. if id=0)
940
    if pinode.setId is 0:
941
      # Retry creation of set if unsuccessful.
942
      pinode.setId = retryFlickrOp(False, self.transfl.createSet,
943
                                   parentPath, inode.photoId)
944
      # If the set is created, then return.
945
      if pinode.setId is not None:
946
        self.updateInode(parentPath, pinode)
947
        return
948
      else:
949
        log.error("Unable to create set:%s"%(parentPath,))
950
        e = OSError("Unable to create set.")
951
        e.errno = EIO
952
        raise e
953
    else:
954
      # If the operation put2Set doesn't throw exception, that means
955
      # that the picture has been successfully added to set.
956
      # Return in that case, retry otherwise.
957
      retryFlickrOp(True, self.transfl.put2Set,
958
                    pinode.setId, inode.photoId)
959
      return
960
961
  #############################
962
  # End of 'handle' Functions.
963
  #############################
964
965
  def write(self, path, buf, off):
966
    log.debug("write:%s:%s"%(path, off))
967
    ind = path.rindex('/')
968
    name_file = path[ind+1:]
969
    if name_file.startswith('.') and name_file.count('.meta')>0:
970
      return self.handleWriteToNonImage(path, buf, off)
971
    typesinfo = mimetypes.guess_type(path)
972
    if typesinfo[0] is None or typesinfo[0].count('image')<=0:
973
      return self.handleWriteToNonImage(path, buf, off)
974
    templist = path.split('/')
975
    inode = None
976
    if path.startswith('/tags'):
977
      e = OSError("Copying to tags not allowed")
978
      e.errno = EIO
979
      raise e
980
    if path.startswith('/stream'):
981
      tags = templist[1].split(':')
982
      templist[1] = tags.pop(0)
983
      path = '/'.join(templist)
984
      inode = self.handleWriteToBuffer(path, buf)
985
      if len(buf) < 4096:
986
        self.handleUploadingImage(path, inode, tags)
987
    elif path.startswith('/sets/'):
988
      setnTags = templist[2].split(':')
989
      setName = setnTags.pop(0)
990
      templist[2] = setName
991
      path = '/'.join(templist)
992
      inode = self.handleWriteToBuffer(path, buf)
993
      if len(buf) < 4096:
994
        templist.pop(-1)
995
        parentPath = '/'.join(templist)
996
        pinode = self.getInode(parentPath)
997
        inode = self.handleUploadingImage(path, inode, setnTags)
998
        self.handleWriteAddToSet(parentPath, pinode, inode)
999
    log.debug("After modifying write:%s:%s"%(path, off))
1000
    if len(buf)<4096:
1001
      templist = path.split('/')
1002
      templist.pop(-1)
1003
      parentPath = '/'.join(templist)
1004
      try:
1005
        self.inodeCache.pop(path)
1006
      except:
1007
        pass
1008
      INFO = self.transfl.getPhotoInfo(inode.photoId)
1009
      info = self.transfl.parseInfoFromFullInfo(inode.photoId, INFO)
1010
      self._mkfileWithMeta(parentPath, info)
1011
      self.writeMetaInfo(inode.photoId, INFO)
1012
    return len(buf)
1013
1014
  def getInode(self, path):
1015
    if self.inodeCache.has_key(path):
1016
      #log.debug("Got cached inode: " + path)
1017
      return self.inodeCache[path]
1018
    else:
1019
      #log.debug("No inode??? I DIE!!!")
1020
      return None
1021
1022
  def updateInode(self, path, inode):
1023
    self.inodeCache[path] = inode
1024
1025
  def release(self, path, flags):
1026
    log.debug("flickrfs.py:Flickrfs:release: %s %s" % (path,flags))
1027
    return 0
1028
  
1029
  def statfs(self):
1030
    """
1031
  Should return a tuple with the following elements in respective order:
1032
  
1033
  F_BSIZE - Preferred file system block size. (int)
1034
  F_FRSIZE - Fundamental file system block size. (int)
1035
  F_BLOCKS - Total number of blocks in the filesystem. (long)
1036
  F_BFREE - Total number of free blocks. (long)
1037
  F_BAVAIL - Free blocks available to non-super user. (long)
1038
  F_FILES - Total number of file nodes. (long)
1039
  F_FFREE - Total number of free file nodes. (long)
1040
  F_FAVAIL - Free nodes available to non-super user. (long)
1041
  F_FLAG - Flags. System dependent: see statvfs() man page. (int)
1042
  F_NAMEMAX - Maximum file name length. (int)
1043
  Feel free to set any of the above values to 0, which tells
1044
  the kernel that the info is not available.
1045
    """
1046
    block_size = 1024
1047
    blocks = 0L
1048
    blocks_free = 0L
1049
    files = 0L
1050
    files_free = 0L
1051
    namelen = 255
1052
    # statfs is being called repeatedly at least once a second.
1053
    # The bandwidth information doesn't change that often, so
1054
    # save upon communication with flickr servers to retrieve this
1055
    # information. Only retrieve it once in a while.
1056
    if self.statfsCounter >= 500 or self.statfsCounter is -1:
1057
      (self.max, self.used) = self.transfl.getBandwidthInfo()
1058
      self.statfsCounter = 0
1059
      log.info('statfs: Retrieved Bandwidth info: max %s used %s' % 
1060
               (self.max, self.used))
1061
    self.statfsCounter = self.statfsCounter + 1
1062
1063
    if self.max is not None:
1064
      blocks = long(self.max)/block_size
1065
      blocks_used = long(self.used)/block_size
1066
      blocks_free = blocks - blocks_used
1067
      blocks_available = blocks_free
1068
      return (block_size, blocks, blocks_free, blocks_available, 
1069
              files, files_free, namelen)
1070
1071
  def fsync(self, path, isfsyncfile):
1072
    log.debug("flickrfs.py:Flickrfs:fsync: path=%s, isfsyncfile=%s" % 
1073
              (path,isfsyncfile))
1074
    return 0
1075
1076
1077
if __name__ == '__main__':
1078
  try:
1079
    server = Flickrfs()
1080
    server.multithreaded = 1;
1081
    server.main()
1082
  except KeyError:
1083
    log.error('Got key error. Exiting...')
1084
    sys.exit(0)
(-)flickrfs-1.3.9.orig/inodes.py (-134 lines)
Lines 1-134 Link Here
1
#===============================================================================
2
#  flickrfs - Virtual Filesystem for Flickr
3
#  Copyright (c) 2005,2006 Manish Rai Jain  <manishrjain@gmail.com>
4
#
5
#  This program can be distributed under the terms of the GNU GPL version 2, or 
6
#  its later versions. 
7
#
8
# DISCLAIMER: The API Key and Shared Secret are provided by the author in 
9
# the hope that it will prevent unnecessary trouble to the end-user. The 
10
# author will not be liable for any misuse of this API Key/Shared Secret 
11
# through this application/derived apps/any 3rd party apps using this key. 
12
#===============================================================================
13
14
__author__ =  "Manish Rai Jain (manishrjain@gmail.com)"
15
__license__ = "GPLv2 (details at http://www.gnu.org/licenses/licenses.html#GPL)"
16
17
import os, sys, time
18
from stat import *
19
import cPickle
20
21
DEFAULTBLOCKSIZE = 4*1024 # 4 KB
22
23
class Inode(object):
24
  """Common base class for all file system objects
25
  """
26
  def __init__(self, path=None, id='', mode=None, 
27
               size=0L, mtime=None, ctime=None):
28
    self.nlink = 1
29
    self.size = size 
30
    self.id = id
31
    self.mode = mode
32
    self.ino = long(time.time())
33
    self.dev = 409089L
34
    self.uid = int(os.getuid())
35
    self.gid = int(os.getgid())
36
    now = int(time.time())
37
    self.atime = now
38
    if mtime is None:
39
      self.mtime = now
40
    else:
41
      self.mtime = int(mtime)
42
    if ctime is None:
43
      self.ctime = now
44
    else:
45
      self.ctime = int(ctime)
46
    self.blocksize = DEFAULTBLOCKSIZE
47
48
class DirInode(Inode):
49
  def __init__(self, path=None, id="", mode=None, mtime=None, ctime=None):
50
    if mode is None: mode = 0755
51
    super(DirInode, self).__init__(path, id, mode, 0L, mtime, ctime)
52
    self.mode = S_IFDIR | self.mode
53
    self.nlink += 1
54
    self.dirfile = ""
55
    self.setId = self.id
56
57
58
class FileInode(Inode):
59
  def __init__(self, path=None, id="", mode=None, comm_meta="", 
60
               size=0L, mtime=None, ctime=None):
61
    if mode is None: mode = 0644
62
    super(FileInode, self).__init__(path, id, mode, size, mtime, ctime)
63
    self.mode = S_IFREG | self.mode
64
    self.photoId = self.id
65
    self.comm_meta = comm_meta
66
67
68
class ImageCache:
69
  def __init__(self):
70
    self.bufDict = {}
71
72
  def setBuffer(self, id, buf):
73
    self.bufDict[id] = buf
74
75
  def addBuffer(self, id, inc):
76
    buf = self.getBuffer(id)
77
    self.setBuffer(id, buf+inc)
78
79
  def getBuffer(self, id, start=0, end=0):
80
    if end == 0:
81
      return self.bufDict.get(id, "")[start:]
82
    else:
83
      return self.bufDict.get(id, "")[start:end]
84
85
  def getBufLen(self, id):
86
    return long(len(self.bufDict.get(id, "")))
87
88
  def popBuffer(self, id):
89
    if id in self.bufDict:
90
      return self.bufDict.pop(id)
91
92
93
class InodeCache(dict):
94
  def __init__(self, dbPath):
95
    dict.__init__(self)
96
    try:
97
      import bsddb
98
      # If bsddb is available, utilize that package
99
      # and store the inodes in database.
100
      self.db = bsddb.btopen(dbPath, flag='c')
101
    except:
102
      # Otherwise, store the inodes in memory.
103
      self.db = {}
104
    # Keep the keys in memory.
105
    self.keysCache = set()
106
  
107
  def __getitem__(self, key, d=None):
108
    # key k may be unicode, so convert it to
109
    # a normal string first.
110
    if not self.has_key(key):
111
      return d
112
    valObjStr = self.db.get(str(key))
113
    return cPickle.loads(valObjStr)
114
115
  def __setitem__(self, key, value):
116
    self.keysCache.add(key)
117
    self.db[str(key)] = cPickle.dumps(value)
118
  
119
  def get(self, k, d=None):
120
    return self.__getitem__(k, d)
121
122
  def keys(self):
123
    return list(self.keysCache)
124
  
125
  def pop(self, k, *args):
126
    # key k may be unicode, so convert it to
127
    # a normal string first.
128
    valObjStr = self.db.pop(str(k), *args)
129
    self.keysCache.discard(k)
130
    if valObjStr != None:
131
      return cPickle.loads(valObjStr)
132
133
  def has_key(self, k):
134
    return k in self.keysCache
(-)flickrfs-1.3.9.orig/README (+12 lines)
Lines 61-63 Link Here
61
            copy of the extenion name tacked on.
61
            copy of the extenion name tacked on.
62
      - bugfix: fixed inability to retrieve certain photos.
62
      - bugfix: fixed inability to retrieve certain photos.
63
      - bugfix: added missing import of 'sys' module (only used during certain error exits)
63
      - bugfix: added missing import of 'sys' module (only used during certain error exits)
64
      - 20080127RDM bugfix: added fuse-python 0.2 compatibility by setting the
65
          python_fuse_api to 0.1.
66
      - 20080128RDM bugfix: use only one thread to populate photos in sets,
67
          rather than one thread per set.  This seems to fix the problem of
68
          having no photos appear when there are lots of sets.
69
      - 20080128RDM bugfix: handle the case of the photo not having an
70
           'originalformat' property in getPhotoInfo (default to jpg).
71
      - 20080129RDM bugfix: removed limit of 500 images per directory
72
           in the 'sets' directory.  Bug still exists for other directories.
73
      - 20080131RDM feature: made the logging message formats more consistent, and
74
           made fuller use of the python logging facility features.
75
      - 20080131RDM bugfix: log exceptions in timerThread if they happen
(-)flickrfs-1.3.9.orig/setup.py (+25 lines)
Line 0 Link Here
1
######################################################################
2
##
3
## Copyright (C) 2006,  Varun Hiremath <varunhiremath@gmail.com>
4
##
5
## Filename:      setup.py
6
## Author:        Varun Hiremath <varunhiremath@gmail.com>
7
## Description:   Installation script
8
## License:       GPL
9
######################################################################
10
11
import os, sys
12
from distutils.core import setup
13
14
PROGRAM_NAME = "flickrfs"
15
PROGRAM_VERSION = "1.3.9"
16
PROGRAM_URL = "http://manishrjain.googlepages.com/flickrfs"
17
18
setup(name='%s' % (PROGRAM_NAME).lower(),
19
      version='%s' % (PROGRAM_VERSION),
20
      description="virtual filesystem for flickr online photosharing service",
21
      author="Manish Rai Jain",
22
      license='GPL-2',
23
      url="%s" % (PROGRAM_URL),
24
      author_email=" <manishrjain@gmail.com>",
25
      packages = ['flickrfs'])
(-)flickrfs-1.3.9.orig/transactions.py (-395 lines)
Lines 1-395 Link Here
1
#===============================================================================
2
#  flickrfs - Virtual Filesystem for Flickr
3
#  Copyright (c) 2005,2006 Manish Rai Jain  <manishrjain@gmail.com>
4
#
5
#  This program can be distributed under the terms of the GNU GPL version 2, or 
6
#  its later versions. 
7
#
8
# DISCLAIMER: The API Key and Shared Secret are provided by the author in 
9
# the hope that it will prevent unnecessary trouble to the end-user. The 
10
# author will not be liable for any misuse of this API Key/Shared Secret 
11
# through this application/derived apps/any 3rd party apps using this key. 
12
#===============================================================================
13
14
__author__ =  "Manish Rai Jain (manishrjain@gmail.com)"
15
__license__ = "GPLv2 (details at http://www.gnu.org/licenses/licenses.html#GPL)"
16
17
from flickrapi import FlickrAPI
18
from traceback import format_exc
19
import urllib2
20
import sys
21
import string
22
import os
23
import time
24
25
# flickr auth information
26
flickrAPIKey = "f8aa9917a9ae5e44a87cae657924f42d"  # API key
27
flickrSecret = "3fbf7144be7eca28"  # shared "secret"
28
29
# Utility functions
30
def kwdict(**kw): return kw
31
32
#Transactions with flickr, wraps FlickrAPI 
33
# calls in Flickfs-specialized functions.
34
class TransFlickr: 
35
36
  extras = "original_format,date_upload,last_update"
37
38
  def __init__(self, logg, browserName):
39
    global log
40
    log = logg
41
    self.fapi = FlickrAPI(flickrAPIKey, flickrSecret)
42
    self.user_id = ""
43
    # proceed with auth
44
    # TODO use auth.checkToken function if available, 
45
    # and wait after opening browser.
46
    print "Authorizing with flickr..."
47
    log.info("Authorizing with flickr...")
48
    try:
49
      self.authtoken = self.fapi.getToken(browser=browserName)
50
    except:
51
      print ("Can't retrieve token from browser:%s:"%(browserName,))
52
      print ("\tIf you're behind a proxy server,"
53
             " first set http_proxy environment variable.")
54
      print "\tPlease close all your browser windows, and try again"
55
      log.error(format_exc())
56
      log.error("Can't retrieve token from browser:%s:"%(browserName,))
57
      sys.exit(-1)
58
    if self.authtoken == None:
59
      log.error('Not able to authorize. Exiting...')
60
      sys.exit(-1)
61
        #Add some authorization checks here(?)
62
    print "Authorization complete."
63
    log.info('Authorization complete')
64
    
65
  def uploadfile(self, filepath, taglist, bufData, mode):
66
    #Set public 4(always), 1(public). Public overwrites f&f.
67
    public = mode&1
68
    #Set friends and family 4(always), 2(family), 1(friends).
69
    friends = mode>>3 & 1
70
    family = mode>>4 & 1
71
      #E.g. 745 - 4:No f&f, but 5:public
72
      #E.g. 754 - 5:friends, but not public
73
      #E.g. 774 - 7:f&f, but not public
74
75
    log.info("Uploading file %s with data of len %s" % (filepath, len(bufData)))
76
    log.info("and tags %s" % str(taglist))
77
    log.info("Permissions: Family %s Friends %s Public %s" % 
78
             (family,friends,public))
79
    filename = os.path.splitext(os.path.basename(filepath))[0]
80
    rsp = self.fapi.upload(filename=filepath, jpegData=bufData,
81
          title=filename,
82
          tags=taglist,
83
          is_public=public and "1" or "0",
84
          is_friend=friends and "1" or "0",
85
          is_family=family and "1" or "0")
86
87
    if rsp is None:
88
      log.error("Seems like we could't write file %s."
89
                " Running checks." % filepath)
90
      recent_rsp = None
91
      trytimes = 2
92
      while(trytimes):
93
        log.info("Trying to retrieve the recently updated photo."
94
                 " Sleeping for 3 seconds...")
95
        time.sleep(3)
96
        trytimes -= 1
97
        # Keep on trying to retrieve the recently uploaded photo, till we
98
        # actually get the information, or the function throws an exception.
99
        while(recent_rsp is None or not recent_rsp):
100
          recent_rsp = self.fapi.photos_recentlyUpdated(
101
              auth_token=self.authtoken, min_date='1', per_page='1')
102
        
103
        pic = recent_rsp.photos[0].photo[0]
104
        log.info('Pic looking for is %s' % filename)
105
        log.info('Most recently updated pic is %s' % pic['title'])
106
        if filename == pic['title']:
107
          id = pic['id']
108
          log.info("File uploaded %s with photoid %s" % (filepath, id))
109
          return id
110
      return None
111
    else:
112
      id = rsp.photoid[0].elementText
113
      log.info("File uploaded %s with photoid %s" % (filepath, id))
114
      return id
115
116
  def put2Set(self, set_id, photo_id):
117
    log.info("Uploading photo %s to set id %s" % (photo_id, set_id))
118
    rsp = self.fapi.photosets_addPhoto(auth_token=self.authtoken, 
119
                                       photoset_id=set_id, photo_id=photo_id)
120
    if rsp:
121
      log.info("Uploaded photo to set")
122
    else:
123
      log.error(rsp.errormsg)
124
  
125
  def createSet(self, path, photo_id):
126
    log.info("Creating set %s with primary photo %s" % (path,photo_id))
127
    path, title = os.path.split(path)
128
    rsp = self.fapi.photosets_create(auth_token=self.authtoken, 
129
                                     title=title, primary_photo_id=photo_id)
130
    if rsp:
131
      log.info("Created set %s" % title)
132
      return rsp.photoset[0]['id']
133
    else:
134
      log.error(rsp.errormsg)
135
  
136
  def deleteSet(self, set_id):
137
    log.info("Deleting set %s" % set_id)
138
    if str(set_id)=="0":
139
      log.info("The set %s is non-existant online." % set_id)
140
      return
141
    rsp = self.fapi.photosets_delete(auth_token=self.authtoken, 
142
                                     photoset_id=set_id)
143
    if rsp:
144
      log.info("Deleted set %s." % set_id)
145
    else:
146
      log.error(rsp.errormsg)
147
  
148
  def getPhotoInfo(self, photoId):
149
    rsp = self.fapi.photos_getInfo(auth_token=self.authtoken, photo_id=photoId)
150
    if not rsp:
151
      log.error("Can't retrieve information about photo %s, with error %s" % 
152
                (photoId, rsp.errormsg))
153
      return None
154
    format = rsp.photo[0]['originalformat']
155
    perm_public = rsp.photo[0].visibility[0]['ispublic']
156
    perm_family = rsp.photo[0].visibility[0]['isfamily']
157
    perm_friend = rsp.photo[0].visibility[0]['isfriend']
158
    if perm_public == '1':
159
      mode = 0755
160
    else:
161
      b_cnt = 4
162
      if perm_family == '1':
163
        b_cnt += 2
164
      if perm_friend == '1':
165
        b_cnt += 1
166
      mode = "07" + str(b_cnt) + "4"
167
      mode = int(mode)
168
      
169
    if hasattr(rsp.photo[0],'permissions'):
170
      permcomment = rsp.photo[0].permissions[0]['permcomment']
171
      permaddmeta = rsp.photo[0].permissions[0]['permaddmeta']
172
    else:
173
      permcomment = permaddmeta = [None]
174
      
175
    commMeta = '%s%s' % (permcomment,permaddmeta) # Required for chmod.
176
    desc = rsp.photo[0].description[0].elementText
177
    title = rsp.photo[0].title[0].elementText
178
    if hasattr(rsp.photo[0].tags[0], "tag"):
179
      taglist = [ a.elementText for a in rsp.photo[0].tags[0].tag ]
180
    else:
181
      taglist = []
182
    license = rsp.photo[0]['license']
183
    owner = rsp.photo[0].owner[0]['username']
184
    ownerNSID = rsp.photo[0].owner[0]['nsid']
185
    url = rsp.photo[0].urls[0].url[0].elementText
186
    posted = rsp.photo[0].dates[0]['posted']
187
    lastupdate = rsp.photo[0].dates[0]['lastupdate']
188
    return (format, mode, commMeta, desc, title, taglist, 
189
            license, owner, ownerNSID, url, int(posted), int(lastupdate))
190
191
  def setPerm(self, photoId, mode, comm_meta="33"):
192
    public = mode&1 #Set public 4(always), 1(public). Public overwrites f&f
193
    #Set friends and family 4(always), 2(family), 1(friends) 
194
    friends = mode>>3 & 1
195
    family = mode>>4 & 1
196
    if len(comm_meta)<2: 
197
      # This wd patch string index out of range bug, caused 
198
      # because some photos may not have comm_meta value set.
199
      comm_meta="33"
200
    rsp = self.fapi.photos_setPerms(auth_token=self.authtoken,
201
                                    is_public=str(public),
202
                                    is_friend=str(friends), 
203
                                    is_family=str(family), 
204
                                    perm_comment=comm_meta[0],
205
                                    perm_addmeta=comm_meta[1], 
206
                                    photo_id=photoId)
207
    if not rsp:
208
      log.error("Couldn't set permission for photo %s with error %s" % 
209
                (photoId,rsp.errormsg))
210
      return False
211
    log.info("Permission has been set for photo %s" % photoId)
212
    return True
213
214
  def setTags(self, photoId, tags):
215
    templist = [ '"%s"'%(a,) for a in string.split(tags, ',')] + ['flickrfs']
216
    tagstring = ' '.join(templist)
217
    rsp = self.fapi.photos_setTags(auth_token=self.authtoken, 
218
                                   photo_id=photoId, tags=tagstring)
219
    if not rsp:
220
      log.error("Couldn't set tags %s with error %s" % 
221
                (photoId, rsp.errormsg))
222
      return False
223
    return True
224
  
225
  def setMeta(self, photoId, title, desc):
226
    rsp = self.fapi.photos_setMeta(auth_token=self.authtoken, 
227
                                   photo_id=photoId, title=title, 
228
                                   description=desc)
229
    if not rsp:
230
      log.error("Couldn't set meta info for photo %s with error" % 
231
                (photoId, rsp.errormsg))
232
      return False
233
    return True
234
235
  def getLicenses(self):
236
    rsp = self.fapi.photos_licenses_getInfo()
237
    if not rsp:
238
      log.error("Couldn't retrieve licenses with error %s" % rsp.errormsg)
239
      return None
240
    licenseDict = {}
241
    for l in rsp.licenses[0].license:
242
      licenseDict[l['id']] = l['name']
243
    keys = licenseDict.keys()
244
    keys.sort()
245
    sortedLicenseList = []
246
    for k in keys:
247
      # Add tuple of license key, and license value.
248
      sortedLicenseList.append((k, licenseDict[k]))
249
    return sortedLicenseList
250
    
251
  def setLicense(self, photoId, license):
252
    rsp = self.fapi.photos_licenses_setLicense(auth_token=self.authtoken, 
253
                                               photo_id=photoId, 
254
                                               license_id=license)
255
    if not rsp:
256
      log.error("Couldn't set license info for photo %s with error %s" % 
257
                (photoId, rsp.errormsg))
258
      return False
259
    return True
260
261
  def getPhoto(self, photoId):
262
    rsp = self.fapi.photos_getSizes(auth_token=self.authtoken, 
263
                                    photo_id=photoId)
264
    if not rsp:
265
      log.error("Error while trying to retrieve size information"
266
                " for photo %s" % photoId)
267
      return None
268
    buf = ""
269
    for a in rsp.sizes[0].size:
270
      if a['label']=='Original':
271
        try:
272
          f = urllib2.urlopen(a['source'])
273
          buf = f.read()
274
        except:
275
          log.error("Exception in getPhoto")
276
          log.error(format_exc())
277
          return ""
278
    if not buf:
279
      f = urllib2.urlopen(rsp.sizes[0].size[-1]['source'])
280
      buf = f.read()
281
    return buf
282
283
  def removePhotofromSet(self, photoId, photosetId):
284
    rsp = self.fapi.photosets_removePhoto(auth_token=self.authtoken, 
285
                                          photo_id=photoId, 
286
                                          photoset_id=photosetId)
287
    if rsp:
288
      log.info("Photo %s removed from set %s" % (photoId, photosetId))
289
    else:
290
      log.error(rsp.errormsg)
291
      
292
    
293
  def getBandwidthInfo(self):
294
    log.debug("Retrieving bandwidth information")
295
    rsp = self.fapi.people_getUploadStatus(auth_token=self.authtoken)
296
    if not rsp:
297
      log.error("Can't retrieve bandwidth information: %s" % rsp.errormsg)
298
      return (None,None)
299
    bw = rsp.user[0].bandwidth[0]
300
    log.debug("Bandwidth: max:" + bw['max'])
301
    log.debug("Bandwidth: used:" + bw['used'])
302
    return (bw['max'], bw['used'])
303
304
  def getUserId(self):
305
    rsp = self.fapi.auth_checkToken(api_key=flickrAPIKey, 
306
                                    auth_token=self.authtoken)
307
    if not rsp:
308
      log.error("Unable to get userid:" + rsp.errormsg)
309
      return None
310
    usr = rsp.auth[0].user[0]
311
    log.info("Got NSID:"+ usr['nsid'] + ":")
312
    #Set self.user_id to this value
313
    self.user_id = usr['nsid']
314
    return usr['nsid']
315
316
  def getPhotosetList(self):
317
    if self.user_id is "":
318
      self.getUserId() #This will set the value of self.user_id
319
    rsp = self.fapi.photosets_getList(auth_token=self.authtoken, 
320
                                      user_id=self.user_id)
321
    if not rsp:
322
      log.error("Error getting photoset list: %s" % (rsp.errormsg))
323
      return []
324
    if not hasattr(rsp.photosets[0], "photoset"):
325
      return []
326
    return rsp.photosets[0].photoset
327
328
  def parseInfoFromPhoto(self, photo, perms=None):
329
    info = {}
330
    info['id'] = photo['id']
331
    info['title'] = photo['title'].replace('/', '_')
332
    info['format'] = photo['originalformat']
333
    info['dupload'] = photo['dateupload']
334
    info['dupdate'] = photo['lastupdate']
335
    info['perms'] = perms
336
    return info
337
338
  def parseInfoFromFullInfo(self, id, fullInfo):
339
    info = {}
340
    info['id'] = id
341
    info['title'] = fullInfo[4]
342
    info['format'] = fullInfo[0]
343
    info['dupload'] = fullInfo[10]
344
    info['dupdate'] = fullInfo[11]
345
    info['mode'] = fullInfo[1]
346
    return info
347
348
  def getPhotosFromPhotoset(self, photoset_id):
349
    photosPermsMap = {}
350
    for i in range(1,6):
351
      rsp = self.fapi.photosets_getPhotos(auth_token=self.authtoken,
352
                                          photoset_id=photoset_id, 
353
                                          extras=self.extras, 
354
                                          privacy_filter=str(i))
355
      if not rsp:
356
        continue
357
      for p in rsp.photoset[0].photo:
358
        photosPermsMap[p] = str(i)
359
    return photosPermsMap
360
            
361
  def getPhotoStream(self, user_id):
362
    retList = []
363
    pageNo = 1
364
    maxPage = 1
365
    while pageNo<=maxPage:
366
      log.info("maxPage:%s pageNo:%s"%(maxPage, pageNo))
367
      rsp = self.fapi.photos_search(auth_token=self.authtoken, 
368
                                    user_id=user_id, per_page="500", 
369
                                    page=str(pageNo), extras=self.extras)
370
      if not rsp:
371
        log.error("Can't retrive photos from your stream: %s" % rsp.errormsg)
372
        return retList
373
      if not hasattr(rsp.photos[0], 'photo'):
374
        log.error("Doesn't have attribute photos. Page requested %s" % pageNo)
375
        return retList
376
      for a in rsp.photos[0].photo:
377
        retList.append(a)
378
      maxPage = int(rsp.photos[0]['pages'])
379
      pageNo = pageNo + 1
380
    return retList
381
 
382
  def getTaggedPhotos(self, tags, user_id=None):
383
    kw = kwdict(auth_token=self.authtoken, tags=tags, tag_mode="all", 
384
                extras=self.extras, per_page="500")
385
    if user_id is not None: 
386
      kw = kwdict(user_id=user_id, **kw)
387
    rsp = self.fapi.photos_search(**kw)
388
    log.debug("Search for photos with tags %s has been"
389
              " successfully finished." % tags)
390
    if not rsp:
391
      log.error("Couldn't search for the photos: %s" % rsp.errormsg)
392
      return
393
    if not hasattr(rsp.photos[0], 'photo'):
394
      return []
395
    return rsp.photos[0].photo

Return to bug 189000