Excessive file access in a perl script? (10)

1 Name: #!/usr/bin/anonymous : 2011-07-22 16:53 ID:5HtMA4vP

I'm working on this perl script for a website - I have basically two types of pages, A and B, each with a title, some text and other details stored in an individual text file. Each A is associated with a number of Bs, up to 100 or so, which are listed on its page. In order to generate the list of B's for an A page it has to open each relevant B file to read the title out of it.

So as it stands, to generate certain A pages it might be opening 100+ different B files. Now I don't know jack shit about servers but I am aware that file access takes time. And given that it's on shared hosting, am I likely to run into problems here when it goes public? Would I be better off putting all the B's in one big file, or splitting them into blocks of say 10-20, or even just biting the bullet and using some SQL database? Or is it fine the way it is?

2 Name: ᴍʀ. ᴠɪʟᴇ!PRoGN.piPE : 2011-07-24 16:24 ID:Heaven

if you want database-like functionality but don't want to use SQL, try using Storable

use Storable;
my %big_hash = (
...
);
store \%big_hash, 'this_file.db';

and to open:

use Storable;
my %hash = %{ retrieve 'this_file.db' };

3 Name: ᴍʀ. ᴠɪʟᴇ!PRoGN.piPE : 2011-07-24 16:25 ID:Heaven

*** DOUBLE POST ***

note that Storable works with scalars and arrays too of course

4 Name: #!/usr/bin/anonymous : 2011-07-24 23:38 ID:5HtMA4vP

>>2-3 Thanks, I'll keep that in mind.

do you know if the way I'm doing it now will cause any significant performance issues though? I'm happy enough sticking with it for now if it works just as well, rather than redoing everything.

5 Name: ᴍʀ. ᴠɪʟᴇ!PRoGN.piPE : 2011-07-27 17:00 ID:Heaven

>>4

you will take a significant performance hit from reading 100+ files every time you generate a page

6 Name: #!/usr/bin/anonymous : 2011-07-27 22:14 ID:Heaven

>>5
ok, thanks. I suspected as much, I was just vainly hoping there might be some obscure reason that it'd all be fine and I wouldn't have to rewrite everything. :D

7 Name: #!/usr/bin/anonymous : 2011-07-28 21:51 ID:Heaven

you should really generate the A pages as static html files and only regenerate them when the content changes.

8 Name: #!/usr/bin/anonymous : 2011-07-31 10:28 ID:Heaven

I agree with >>7

9 Name: 1 : 2011-08-01 04:57 ID:Heaven

>>7-8 yeah that's a good plan actually. I may well do that.

10 Name: #!/usr/bin/anonymous : 2012-01-15 16:18 ID:2W+UB1mm

Just curious: I have a similar type of site as >>1 but it uses a MySQL database, is it worth doing something like >>7 suggested with that too?

This thread has been closed. You cannot post in this thread any longer.