96902 – bacula seems to get into an infinite loop when jobs have many files

Bug 96902 - bacula seems to get into an infinite loop when jobs have many files

Summary: bacula seems to get into an infinite loop when jobs have many files

Status:	RESOLVED NEEDINFO

Alias:	None

Product:	Gentoo Linux
Classification:	Unclassified
Component:	Current packages (show other bugs)
Hardware:	x86 Linux

Importance:	High major (vote)
Assignee:	Chris Lee (RETIRED)

URL:
Whiteboard:
Keywords:

Depends on:
Blocks:

Reported:	2005-06-23 15:59 UTC by Jeffrey Crawford
Modified:	2006-08-08 04:43 UTC (History)
CC List:	1 user (show)

See Also:
Package list:
Runtime testing required:	---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description Jeffrey Crawford 2005-06-23 15:59:31 UTC

When a job has many files, In this example around 413000000, bacula-dir seems to
hang after the files have been retrieved. bacula-dir will run at 100% CPU
utilization and the construction of the interactive file selection application
in bconsole no longer responds. This does not appear to be due to any corruption
as I can restore files if I know the exact name and path. However when files are
not known the interactive restore console is essential. Here is an example of
the results:

akebono root # /etc/bacula/bconsole
Connecting to Director akebono:9101
1000 OK: akebono-dir Version: 1.36.0 (20 October 2004)
Enter a period to cancel a command.
*restore
Using default Catalog name=ProTradeCatalog DB=bacula

First you select one or more JobIds that contain files
to be restored. You will be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.

To select the JobIds, you have the following choices:
     1: List last 20 Jobs run
     2: List Jobs where a given File is saved
     3: Enter list of comma separated JobIds to select
     4: Enter SQL list command
     5: Select the most recent backup for a client
     6: Select backup for a client before a specified time
     7: Enter a list of files to restore
     8: Enter a list of files to restore before a specified time
     9: Cancel
Select item:  (1-9): 5
Defined Clients:
     1: backup-fd
     2: practice1-fd
     3: brady-fd
     4: beckham-fd
     5: ali-fd
     6: pele-fd
     7: stage1-fd
     8: stage2-fd
     9: kwan-fd
    10: aaron-fd
    11: vpn-fd
    12: fw-fd
    13: akebono-fd
Select the Client (1-13): 3
Automatically selected FileSet: BeckhamBradyFiles
+-------+-------+-----------+---------------------+-------------------+-----------+--------------+----------------+
| jobid | level | jobfiles  | starttime           | volumename        |
startfile | volsessionid | volsessiontime |
+-------+-------+-----------+---------------------+-------------------+-----------+--------------+----------------+
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
343 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
342 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
341 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
340 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
339 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
338 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
337 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
336 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
335 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
334 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
333 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
332 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
331 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
330 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
329 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
328 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
327 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
326 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
325 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
324 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
323 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
322 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
321 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
320 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
319 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
318 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
317 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
316 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
315 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
314 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
313 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
312 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
311 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
310 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
309 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
308 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
307 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
306 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
305 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
304 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
303 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
302 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
301 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
300 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
299 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
298 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
297 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
296 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
295 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
294 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
293 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
292 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
291 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
290 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
289 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
288 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
287 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
286 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
285 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
284 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
283 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
282 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
281 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
280 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
279 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
278 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
277 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
276 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
275 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
274 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
273 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
272 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
271 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
270 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
269 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
268 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
267 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
266 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
265 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
264 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
263 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
262 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
261 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
260 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
259 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
258 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
257 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
256 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
255 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
254 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
253 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
252 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
251 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
250 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
249 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
248 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
247 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
246 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
245 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
244 |          195 |  1,116,014,565 |
| 2,220 | F     | 4,130,252 | 2005-06-03 22:00:01 | ProTrade_Tape-001 |      
243 |          195 |  1,116,014,565 |
| 2,366 | D     |   125,844 | 2005-06-17 22:00:00 | ProTrade_Tape-004 |      
221 |           13 |  1,118,965,859 |
| 2,377 | I     |       903 | 2005-06-18 22:00:01 | ProTrade_Tape-004 |      
312 |           24 |  1,118,965,859 |
| 2,388 | I     |     6,354 | 2005-06-19 22:00:00 | ProTrade_Tape-005 |       
13 |           35 |  1,118,965,859 |
| 2,399 | I     |    66,749 | 2005-06-20 22:00:01 | ProTrade_Tape-005 |       
58 |           46 |  1,118,965,859 |
| 2,399 | I     |    66,749 | 2005-06-20 22:00:01 | ProTrade_Tape-005 |       
57 |           46 |  1,118,965,859 |
| 2,399 | I     |    66,749 | 2005-06-20 22:00:01 | ProTrade_Tape-005 |       
56 |           46 |  1,118,965,859 |
| 2,399 | I     |    66,749 | 2005-06-20 22:00:01 | ProTrade_Tape-005 |       
55 |           46 |  1,118,965,859 |
| 2,410 | I     |    44,629 | 2005-06-21 22:00:01 | ProTrade_Tape-005 |      
102 |           57 |  1,118,965,859 |
| 2,410 | I     |    44,629 | 2005-06-21 22:00:01 | ProTrade_Tape-005 |      
101 |           57 |  1,118,965,859 |
| 2,410 | I     |    44,629 | 2005-06-21 22:00:01 | ProTrade_Tape-005 |      
100 |           57 |  1,118,965,859 |
| 2,410 | I     |    44,629 | 2005-06-21 22:00:01 | ProTrade_Tape-005 |       
99 |           57 |  1,118,965,859 |
| 2,422 | I     |   177,193 | 2005-06-22 22:00:01 | ProTrade_Tape-005 |      
151 |            2 |  1,119,483,884 |
| 2,422 | I     |   177,193 | 2005-06-22 22:00:01 | ProTrade_Tape-005 |      
150 |            2 |  1,119,483,884 |
| 2,422 | I     |   177,193 | 2005-06-22 22:00:01 | ProTrade_Tape-005 |      
149 |            2 |  1,119,483,884 |
| 2,422 | I     |   177,193 | 2005-06-22 22:00:01 | ProTrade_Tape-005 |      
148 |            2 |  1,119,483,884 |
| 2,422 | I     |   177,193 | 2005-06-22 22:00:01 | ProTrade_Tape-005 |      
147 |            2 |  1,119,483,884 |
+-------+-------+-----------+---------------------+-------------------+-----------+--------------+----------------+
You have selected the following JobIds: 2220,2366,2377,2388,2399,2410,2422

Building directory tree for JobId 2220 ...  ++++++++++++++++++++akebono root #

Reproducible: Always
Steps to Reproduce:
1. Set up a very large backup job in this case over 413000000 files
2. start bconsole and start a restore building a directory tree from the big job
3. After a few minutes of database access bconsole hangs and bacula-dir is
running at 100%



Expected Results:  
Interactive file selection application should have started.

There are other jobs on this system that do not exibit this problem other jobs
will allow me to select files and restore using the interactive restore
application. It only shows up with this very big job...

Comment 1 Arnaud Launay 2005-08-09 07:25:06 UTC

Same remark, would you give 1.36.3 a try ? Thanks.

Comment 2 Chris Lee (RETIRED) gentoo-dev

2005-11-22 14:59:26 UTC

As Arnaud suggested, can you reproduce this with 1.36.3?

Comment 3 Fabian Groffen gentoo-dev

2006-08-08 04:43:24 UTC

is this problem still apparent in 1.38.9?