| Summary: | net-proxy/squid-3.3.12 does not cache downloads larger than maximum_object_size_in_memory | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Matthew Stapleton <matthew4196> |
| Component: | [OLD] Server | Assignee: | No maintainer - Look at https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers if you want to take care of it <maintainer-needed> |
| Status: | RESOLVED TEST-REQUEST | ||
| Severity: | minor | CC: | bkohler, net-proxy+disabled |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | AMD64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
|
Description
Matthew Stapleton
2014-08-25 10:04:31 UTC
On a non-hardened test system with empty squid cache, I can't seem to cache any files larger than 4194304 with squid-3.3.12 no matter how high I set maximum_object_size or maximum_object_size_in_memory to. Testing was done by downloading from a local Apache server with a file created with dd. Test squid.conf slightly modified from default: acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl Safe_ports port 901 # SWAT acl CONNECT method CONNECT http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager http_access allow localnet http_access allow localhost http_access deny all http_port 3128 cache_dir aufs /var/cache/squid 2048 16 256 coredump_dir /var/cache/squid refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 reload-into-ims refresh-ims maximum_object_size_in_memory 600 MB maximum_object_size 600 MB via off forwarded_for delete emerge --info: Portage 2.2.8-r1 (default/linux/amd64/13.0/desktop/kde, gcc-4.6.3, glibc-2.15-r3, 3.15.6-gentoo x86_64) ================================================================= System uname: Linux-3.15.6-gentoo-x86_64-Intel-R-_Core-TM-_i7-4700HQ_CPU_@_2.40GHz-with-gentoo-2.1 KiB Mem: 16317388 total, 712736 free KiB Swap: 16777212 total, 16380240 free Timestamp of tree: Sun, 24 Aug 2014 13:15:01 +0000 ld GNU ld (GNU Binutils) 2.22 app-shells/bash: 4.2_p37 dev-java/java-config: 2.1.11-r3 dev-lang/python: 2.6.8, 2.7.5-r2, 3.1.2-r3, 3.2.5-r2, 3.3.2-r2 dev-util/cmake: 2.8.11.2 dev-util/pkgconfig: 0.28-r1 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.12.4 sys-apps/sandbox: 2.6-r1 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.8.5-r4, 1.9.6-r3, 1.10.3, 1.11.1, 1.12.6, 1.13.4 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.4.5, 4.5.3-r1, 4.6.3 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r4 sys-kernel/linux-headers: 3.9 (virtual/os-headers) sys-libs/glibc: 2.15-r3 Repositories: gentoo bumblebee x-portage ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=corei7-avx -fomit-frame-pointer -mmmx -msse3 -msse4.1 -ftree-vectorize -fpredictive-commoning -fno-tree-vect-loop-version -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/maven-bin-2.2/conf /usr/share/maven-bin-3.0/conf /usr/share/themes/oxygen-gtk/gtk-2.0 /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -march=corei7-avx -fomit-frame-pointer -mmmx -msse3 -msse4.1 -ftree-vectorize -fpredictive-commoning -fno-tree-vect-loop-version -pipe" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://mirror.internode.on.net/pub/gentoo http://mirror.3fl.net.au/pub/gentoo ftp://gentoo.mirrors.pair.com/ ftp://mirrors.tds.net/gentoo http://mirrors.tds.net/gentoo" LANG="en_AU.utf8" LDFLAGS="-Wl,--as-needed" MAKEOPTS="-j4" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/bumblebee /usr/local/portage" SYNC="rsync://rsync/gentoo-portage" USE="X a52 aac acl acpi alsa amd64 apache2 arts avahi berkdb bluetooth branding bzip2 cairo caps cdda cdr cjk cli consolekit cracklib crypt cups cxx dbus declarative divx4linux dlloader dri dts dv dvd dvdr emboss encode exif faac fam firefox flac fortran gdbm gif gnome gnutls gpm gstreamer gtk guile hal howl htmlhandbook iconv ieee1394 ipv6 jikes joystick jpeg jpeg2k kde kdeenablefinal kdehiddenvisibility kipi lcms ldap libnotify logrotate mad matroska mmx mng modules mp3 mp4 mpeg multilib nas ncurses net nls nptl odbc ogg openexr opengl openmp pam pango pcre pdf phonon physfs plasma png policykit povray ppds pulseaudio qt3support qt4 readline samba scanner sdl semantic-desktop session slp speex spell sse sse2 sse3 ssl ssse3 startup-notification svg tcpd tetex theora threads tiff truetype udev udisks unicode upower usb vorbis wmf wxwidgets x264 xattr xcb xcomposite xinerama xml xml2 xscreensaver xv xvid zeroconf zlib" ABI_X86="32 64" ALSA_CARDS="intel8x0 intel8x0m via82xx usb-audio hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DVB_CARDS="usb-a800" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev keyboard mouse synaptics linuxinput ps2mouse joystick wacom" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB en_US en_AU zh_CN" LIRC_DEVICES="devinput mceusb mceusb2 atilibusb atiusb" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="fbdev nv v4l vesa nvidia nouveau intel radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON I don't have the 4MB cache problem on the server that runs hardened (The first config posted) though even with a fresh cache folder. On that system if I set maximum_object_size_in_memory 500 MB, I can download a 200MB file and it is stored to the cache. I believe you need to set maximum_object_size before your cache_dir line, can you see if that helps? Okay that's fixed the problem thanks. This means that squid.conf.documented is incorrect as it has maximum_object_size below cache_dir and even though 3.1.8 doesn't have a problem with that, 3.2 series and 3.3 series do have a problem. What I didn't mention is the first config posted is based on squid.conf.documented and then I removed all the comments with "grep -v '^#' /etc/squid/squid.conf | grep '[a-zA-Z0-9]'" . Would this be classed as a bug in squid.conf.documented or in the Squid source code? please retry with net-proxy/squid-3.5.28 |
