View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0008730 | Scribus | Graphics / Image Frames | public | 2010-01-18 22:46 | 2011-01-09 22:25 |
Reporter | mhx | Assigned To | jghali | ||
Priority | normal | Severity | feature | Reproducibility | always |
Status | closed | Resolution | fixed | ||
Fixed in Version | 1.5.0svn | ||||
Summary | 0008730: [PATCH] Add caching for low resolution images | ||||
Description | The attached patch against trunk introduces an image caching framework for Scribus and uses it to cache low resolution preview images. The main objective is to speed up (re-)loading of documents. While working on a large book, I spent 20 minutes just for loading that document into Scribus, and about 10 minutes turning preview mode or color management on/off. With the image cache, this time is shortened by up to a factor of 50. I have tried to ensure that there is absolutely no visual difference between having the cache enabled or disabled. The cache can be disabled and configured in the Scribus preferences. A high level description as well as the main interface documentation is available in doxygen format as part of the patch. | ||||
Tags | No tags attached. | ||||
Patch | |||||
2010-01-18 22:46
|
scribus-trunk-imagecache-14-svn.diff (148,499 bytes)
Index: scribus/scimagecachewriteaction.h =================================================================== --- scribus/scimagecachewriteaction.h (revision 0) +++ scribus/scimagecachewriteaction.h (revision 0) @@ -0,0 +1,60 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#ifndef SCIMAGECACHEWRITEACTION_H +#define SCIMAGECACHEWRITEACTION_H + +#include <QHash> +#include <QString> +#include <QStringList> + +#include "scribusapi.h" +#include "scimagecachedir.h" + +class ScLockedFile; + +/** + * @brief Bracket for write accesses to the image cache + * @author Marcus Holland-Moritz + */ +class ScImageCacheWriteAction +{ +public: + ScImageCacheWriteAction(bool haveMasterLock = false); + ~ScImageCacheWriteAction(); + + bool start(); + bool add(const QString & file); + bool commit(); + +private: + typedef ScImageCacheDir::AccessCounter AccessCounter; + typedef QHash<QString, ScLockedFile *> FileMap; + QStringList m_files; + FileMap m_access; + bool m_locked; + const bool m_haveMasterLock; + + void clear(); + bool unlock(); + bool update(const QString & dir, ScLockedFile *p, AccessCounter & from, AccessCounter & to); +}; + +#endif Index: scribus/scimage.h =================================================================== --- scribus/scimage.h (revision 14532) +++ scribus/scimage.h (working copy) @@ -30,6 +30,8 @@ class ScribusDoc; class ScStreamFilter; class CMSettings; +class ScImageCacheProxy; +class ScColorProfile; class SCRIBUS_API ScImage : private QImage { @@ -94,7 +96,7 @@ void applyEffect(const ScImageEffectList& effectsList, ColorList& colors, bool cmyk); // Generate a low res image for user preview - void createLowRes(double scale); + bool createLowRes(double scale); // Scale this image in-place void scaleImage(int width, int height); @@ -106,6 +108,8 @@ // Load an image into this ScImage instance // TODO: document params, split into smaller functions bool loadPicture(const QString & fn, int page, const CMSettings& cmSettings, RequestType requestType, int gsRes, bool *realCMYK = 0, bool showMsg = false); + bool loadPicture(ScImageCacheProxy & cache, bool & fromCache, int page, const CMSettings& cmSettings, RequestType requestType, int gsRes, bool *realCMYK = 0, bool showMsg = false); + bool saveCache(ScImageCacheProxy & cache); ImageInfoRecord imgInfo; @@ -128,6 +132,8 @@ bool convolveImage(QImage *dest, const unsigned int order, const double *kernel); int getOptimalKernelWidth(double radius, double sigma); void applyCurve(const QVector<int>& curveTable, bool cmyk); + + void addProfileToCacheModifiers(ScImageCacheProxy & cache, const QString & prefix, const ScColorProfile & profile) const; }; #endif Index: scribus/scimagecachedir.h =================================================================== --- scribus/scimagecachedir.h (revision 0) +++ scribus/scimagecachedir.h (revision 0) @@ -0,0 +1,86 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#ifndef SCIMAGECACHEDIR_H +#define SCIMAGECACHEDIR_H + +#include <QObject> +#include <QHash> +#include <QString> +#include <QStringList> + +#include "scribusapi.h" + +class ScImageCacheFile; +class QFileInfo; + +/** + * @brief Representation of a directory node in the image cache tree + * @author Marcus Holland-Moritz + */ +class ScImageCacheDir : public QObject +{ + Q_OBJECT + +public: + typedef unsigned int AccessCounter; + + ScImageCacheDir(const QString & dir, ScImageCacheDir *parent = 0, bool scanFiles = false, const QStringList & suffixList = QStringList()); + ~ScImageCacheDir(); + ScImageCacheDir *newSubDir(const QString & dir, bool scanFiles = false, const QStringList & suffixList = QStringList()); + const QString & name() const { return m_name; } + QString path(bool relative = false) const; + QString path(const QString & file) const; + qint64 size() const; + bool exists() const; + void update(); + bool updateFile(const QString & file); + bool updateAccess(const QString & dir, AccessCounter from, AccessCounter to); + + static bool lastAccess(const QString & dir, AccessCounter & access); + + static const QString accessFileName; + +signals: + void fileCreated(ScImageCacheFile * file, const QFileInfo & info); + void fileChanged(ScImageCacheFile * file, const QFileInfo & info); + void fileRemoved(ScImageCacheFile * file); + +private: + typedef QHash<QString, ScImageCacheDir *> DirMap; + typedef QHash<QString, ScImageCacheFile *> FileMap; + + void scan(); + bool addDir(ScImageCacheDir *dir); + bool isModified(); + bool updateFile(const QStringList & parts, int level = 0); + bool updateAccess(const QStringList & parts, AccessCounter from, AccessCounter to, int level = 0); + + QString m_name; + const QStringList m_suffix; + ScImageCacheDir * const m_parent; + bool m_exists; + bool m_lastAccessValid; + AccessCounter m_lastAccess; + DirMap *m_dirs; + FileMap *m_files; +}; + +#endif Index: scribus/scribuscore.cpp =================================================================== --- scribus/scribuscore.cpp (revision 14532) +++ scribus/scribuscore.cpp (working copy) @@ -29,6 +29,7 @@ #include "filewatcher.h" #include "pluginmanager.h" #include "prefsmanager.h" +#include "scimagecachemanager.h" #include "scpaths.h" #include "scribusapp.h" #include "scribuscore.h" @@ -194,6 +195,15 @@ m_HaveCMS = false; getCMSProfiles(showProfileInfo); initCMS(); + + setSplashStatus( tr("Initializing Image Cache") ); + ScImageCacheManager & icm = ScImageCacheManager::instance(); + icm.setEnabled(prefsManager->appPrefs.imageCachePrefs.cacheEnabled); + icm.setMaxCacheSizeMiB(prefsManager->appPrefs.imageCachePrefs.maxCacheSizeMiB); + icm.setMaxCacheEntries(prefsManager->appPrefs.imageCachePrefs.maxCacheEntries); + icm.setCompressionLevel(prefsManager->appPrefs.imageCachePrefs.compressionLevel); + icm.initialize(); + /* initPalettes(); Index: scribus/scimagecachedir.cpp =================================================================== --- scribus/scimagecachedir.cpp (revision 0) +++ scribus/scimagecachedir.cpp (revision 0) @@ -0,0 +1,333 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#include <QDateTime> +#include <QFile> +#include <QFileInfo> +#include <QDirIterator> +#include <QString> + +#include "sclockedfile.h" +#include "scimagecachedir.h" +#include "scimagecachefile.h" +#include "scimagecachemanager.h" +#include "scpaths.h" + +#define SC_DEBUG_FILE defined(DEBUG_SCIMAGECACHE) +#include "scdebug.h" + +const QString ScImageCacheDir::accessFileName("access"); + +ScImageCacheDir::ScImageCacheDir(const QString & dir, ScImageCacheDir *parent, bool scanFiles, const QStringList & suffixList) + : m_name(dir), m_suffix(suffixList), m_parent(parent), m_exists(false), m_lastAccessValid(false), m_dirs(0), m_files(0) +{ + while (m_name.endsWith('/')) + m_name.chop(1); + if (m_parent) + m_parent->addDir(this); + if (scanFiles) + { + m_files = new FileMap; + Q_CHECK_PTR(m_files); + } +} + +ScImageCacheDir *ScImageCacheDir::newSubDir(const QString & dir, bool scanFiles, const QStringList & suffixList) +{ + ScImageCacheDir *d = new ScImageCacheDir(dir, this, scanFiles, suffixList); + Q_CHECK_PTR(d); + return d; +} + +ScImageCacheDir::~ScImageCacheDir() +{ + if (m_dirs) + { + foreach (ScImageCacheDir *p, *m_dirs) + delete p; + delete m_dirs; + } + if (m_files) + { + foreach (ScImageCacheFile *p, *m_files) + delete p; + delete m_files; + } +} + +QString ScImageCacheDir::path(bool relative) const +{ + QString parent; + if (m_parent) + parent = m_parent->path(relative); + else if (relative) + return QString(); + return parent.isEmpty() ? m_name : parent + "/" + m_name; +} + +QString ScImageCacheDir::path(const QString & file) const +{ + return path() + "/" + file; +} + +bool ScImageCacheDir::isModified() +{ + if (!QFile::exists(path())) + { + if (m_exists) + { + m_exists = false; + m_lastAccessValid = false; + return true; + } + return false; + } + m_exists = true; + AccessCounter access; + bool valid = lastAccess(path(true), access); + bool modified = true; + if (m_lastAccessValid && valid && access == m_lastAccess) + modified = false; + m_lastAccess = access; + m_lastAccessValid = valid; + return modified; +} + +bool ScImageCacheDir::addDir(ScImageCacheDir *dir) +{ + if (!m_dirs) + { + m_dirs = new DirMap; + Q_CHECK_PTR(m_dirs); + if (!m_dirs) + return false; + } + if (m_dirs->contains(dir->name())) + return false; + m_dirs->insert(dir->name(), dir); + return true; +} + +void ScImageCacheDir::update() +{ + if (isModified()) + { + scDebug() << path() << "is modified"; + if (m_dirs) + foreach (ScImageCacheDir *p, *m_dirs) + p->update(); + if (m_files) + scan(); + } +} + +void ScImageCacheDir::scan() +{ + if (!m_exists) + { + if (!m_files->isEmpty()) + { + foreach (ScImageCacheFile *p, *m_files) + { + emit fileRemoved(p); + delete p; + } + m_files->clear(); + } + return; + } + + QDirIterator di(path()); + QHash<QString, bool> current; + + while (di.hasNext()) + { + di.next(); + + QFileInfo info = di.fileInfo(); + + if (info.isFile() && m_suffix.contains(info.suffix())) + { + FileMap::iterator i = m_files->find(info.fileName()); + if (i != m_files->end()) + { + // possibly changed file + ScImageCacheFile *p = *i; + if (p->hasChanged(info)) + emit fileChanged(p, info); + } + else + { + // newly created file + ScImageCacheFile *p = new ScImageCacheFile(info.fileName(), this); + Q_CHECK_PTR(p); + if (p == 0) + return; + emit fileCreated(p, info); + m_files->insert(info.fileName(), p); + } + current[info.fileName()] = true; + } + } + + FileMap::iterator i = m_files->begin(); + while (i != m_files->end()) + { + if (!current.contains(i.key())) + { + // removed file + ScImageCacheFile *p = *i; + i = m_files->erase(i); + emit fileRemoved(p); + delete p; + + } + else + i++; + } +} + +bool ScImageCacheDir::updateFile(const QString & file) +{ + return updateFile(file.split('/')); +} + +bool ScImageCacheDir::updateFile(const QStringList & parts, int level) +{ + if (level < parts.count() - 1) + { + if (!m_dirs) + { + scDebug() << "no subdirs in" << path(); + return false; + } + const QString & dir = parts[level]; + DirMap::iterator i = m_dirs->find(dir); + if (i == m_dirs->end()) + { + scDebug() << "no directory" << dir << "in" << path() << "updateFile" << parts << level; + return false; + } + return (*i)->updateFile(parts, level + 1); + } + if (!m_files) + { + scDebug() << "no m_files in" << path(); + return false; + } + const QString & file = parts[level]; + FileMap::iterator i = m_files->find(file); + QFileInfo info(path(file)); + if (i != m_files->end()) + { + ScImageCacheFile *p = *i; + if (info.exists()) + { + // update + if (p->hasChanged(info)) + emit fileChanged(p, info); + } + else + { + // remove + m_files->erase(i); + emit fileRemoved(p); + delete p; + } + } + else + { + if (info.exists()) + { + // create + ScImageCacheFile *p = new ScImageCacheFile(file, this); + Q_CHECK_PTR(p); + if (p == 0) + return false; + emit fileCreated(p, info); + m_files->insert(file, p); + } + else + { + // Don't make an assertion here. This *can* be a bug, but it + // can also happen if someone else messes with the cache. + scDebug() << "BUG: invalid update request for" << path(file); + return false; + } + } + return true; +} + +bool ScImageCacheDir::updateAccess(const QString & dir, AccessCounter from, AccessCounter to) +{ + return updateAccess(dir.split('/'), from, to); +} + +bool ScImageCacheDir::updateAccess(const QStringList & parts, AccessCounter from, AccessCounter to, int level) +{ + if (level < parts.count()) + { + if (!m_dirs) + { + scDebug() << "no subdirs in" << path(); + return false; + } + const QString & dir = parts[level]; + DirMap::iterator i = m_dirs->find(dir); + if (i == m_dirs->end()) + { + scDebug() << "no directory" << dir << "in" << path() << "updateAccess" << parts << level; + return false; + } + return (*i)->updateAccess(parts, from, to, level + 1); + } + if (!m_lastAccessValid) + { + scDebug() << "cannot update access counter for" << path() << "(lastAccess invalid)"; + return false; + } + if (m_lastAccess != from) + { + scDebug() << "cannot update access counter for" << path() << "(lastAccess =" << m_lastAccess << ", from =" << from << ", to =" << to << ")"; + return false; + } + scDebug() << "updating access counter for" << path() << "from" << from << "to" << to; + m_lastAccess = to; + return true; +} + +bool ScImageCacheDir::lastAccess(const QString & dir, AccessCounter & access) +{ + ScLockedFileRO ro(ScImageCacheManager::absolutePath(dir) + "/" + accessFileName); + if (!ro.exists()) + { + scDebug() << ro.name() << "does not exist"; + return false; + } + if (!ro.open()) + { + scDebug() << "failed to open" << ro.name() << "for reading"; + return false; + } + QTextStream in(ro.io()); + in >> access; + return true; +} + Index: scribus/scimagecachefile.h =================================================================== --- scribus/scimagecachefile.h (revision 0) +++ scribus/scimagecachefile.h (revision 0) @@ -0,0 +1,61 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#ifndef SCIMAGECACHEFILE_H +#define SCIMAGECACHEFILE_H + +#include <QObject> +#include <QDateTime> +#include <QFileInfo> +#include <QString> + +#include "scribusapi.h" + +class ScImageCacheDir; + +/** + * @brief Representation of a file node in the image cache tree + * @author Marcus Holland-Moritz + */ +class ScImageCacheFile : public QObject +{ + Q_OBJECT + +public: + ScImageCacheFile(const QString & name, ScImageCacheDir *parent = 0); + ~ScImageCacheFile(); + + QString path(bool relative = false) const; + qint64 size() const; + const QDateTime & modified() const { return m_modified; }; + bool exists() const; + bool hasChanged(const QFileInfo & info) const; + bool hasChanged() const; + bool update(const QFileInfo & info); + bool update(); + +private: + const QString m_name; + ScImageCacheDir * const m_parent; + QDateTime m_modified; + qint64 m_size; +}; + +#endif Index: scribus/scimagecachemanager.cpp =================================================================== --- scribus/scimagecachemanager.cpp (revision 0) +++ scribus/scimagecachemanager.cpp (revision 0) @@ -0,0 +1,1018 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#include <QDateTime> +#include <QDir> +#include <QDirIterator> +#include <QFileInfo> +#include <QTemporaryFile> + +#include "sclockedfile.h" +#include "scimagecachedir.h" +#include "scimagecachefile.h" +#include "scimagecachemanager.h" +#include "scimagecacheproxy.h" +#include "scimagecachewriteaction.h" +#include "scpaths.h" + +#define SC_DEBUG_FILE defined(DEBUG_SCIMAGECACHE) +#include "scdebug.h" + +/*! + +\page imagecache Scribus Image Cache + +This page gives some details about the Scribus image cache manager implemented +in ScImageCacheManager. The image cache manager, accompanied by a number of +helper classes, is responsible for caching low-resolution versions of images +used in Scribus documents. + +As the loading of images and their conversion to low resolution consumes +a lot of time, the image cache helps to massively speed up the loading of +images that have been previously loaded under the same conditions. It will +also speed up operations like undoing or redoing image effects. + +The image cache was designed to be accessible simultaneously by multiple +instances of Scribus. It should even be possible to share the cache over +a network drive, although this will surely degrade performance. + + +\section ic_filetypes File Types in the Image Cache + +All files stored in the cache are either short XML documents or real image +files. PNG has been chosen as the image format, as it offers good compression +and is a lossless format. At low compression levels, it is also quite fast. + +There are quite a lot of properties in Scribus that have an influence on +how an image will be rendered on the screen. These are mainly color management +and image effects. These properties will be called modifiers in this text. + +Image information is properties directly associated with the on-disk image +file, e.g. resolution or EXIF data. As the original image is not read when +fetching images from the cache, this data needs to be cached as well. + +Meta information, finally, is information describing the cache entry. It +contains properties like the the on-disk image file path, the image file size +or the last modification date. It is used to identify whether or not an image +can be fetched from the cache or must be reloaded from its original file. + +\verbatim +------------------------------------------------------------------------------- + + Meta File (.xml) Reference File (.ref) Image File (.png) + + .-----------------. .-----------------. .-----------------. + |meta information |-------->|reference count | |cached image | + |modifiers | | |-------->| | + |image information| .--->| | | | + '-----------------' | '-----------------' '-----------------' + | + .-----------------. | + |meta information | | + |modifiers |----' + |image information| + '-----------------' + +------------------------------------------------------------------------------- +\endverbatim + +As different combinations of modifiers \em may end up producing exactly the same +image in the cache, multiple meta files may reference the same image file. To +keep track of the number of references, each cached image is accompanied by a +reference file that differs from the image file only by the file suffix. + +To avoid races between multiple instances of Scribus, possibly even running on +different machines, all files must be accessed atomically when the cache is +being modified. Non-modifying, read-only accesses are always allowed. + +This is usually achieved by the following mechanisms: + + - Lock files (with an additional suffix ".lock") are created by the instance + that wishes to modify an entry in the cache. Only the instance that has + successfully acquired all necessary lock files may modify the cache. + As it is close to impossible to atomically create a \em file in a + platform-independent way, the lock file is actually implemented as a lock + directory. See ScLockedFile for details. + + - All files that are created or modified are created as temporary files first. + Only when they have been written completely, the old version of the file is + unlinked and the new version is renamed to its final name. + +This ensures that an instance that only wishes to read from the cache can +safely do so even without caring about locked files. + +Furthermore, in order to avoid any deadlocks or delays, locking only makes sure +that only one instance writes to the cache at a time. If another instance fails +to get the necessary locks, it will simply not not carry out the whole cache +access. + + +\section ic_dirstructure Directory Structure + +Each cache file is uniquely indentified by a hexadecimal MD5 hash. The first +two hex digits represent two levels of subdirectories and the remainder forms +the start of the file's basename, for example: + +\verbatim + $(HOME)/.scribus/cache/img/a/e/15c5160668926e4a7c593a813a0d68.xml +\endverbatim + +Within each folder of the cache structure, there is an additional \c access +file that keeps track of write accesses to this folder. The purpose of this +file is to notify other instances of Scribus when entries in the cache have +been modified. The file simply contains a counter that is incremented with +each write access to the cache. The file also serves as a lock for the +directory. Instead of locking individual files in the cache, locking the +\c access files is sufficient. + + +\section ic_housekeeping Cache Housekeeping + +Each instance doing any write access to the cache will first create its own +lock file in: + +\verbatim + $SCRIBUS/cache/img/locks/ +\endverbatim + +The name does not matter. After successful creation of this file, the instance +checks for the presence of the master lock file + +\verbatim + $SCRIBUS/cache/img/locks/master.lock +\endverbatim + +If this is present, the instance will remove its own lock file and will not +initiate any write accesses to the cache. + +An instance wishing to do a cleanup will attempt to create the master lock +file. If it succeeds, it will check that no other lock files are present in +the lock directory. If other lock files are present, it will remove the master +lock file and not perform a cleanup. If no other lock files are present, the +instance has exclusive write access to all cache files. + +After each write operation, a cache cleanup is performed if necessary. This +means, if the cache limits (number of meta files or total cache size) are +exceeded, the oldest meta files will be removed until the cache is within +the user defined limits again. + +In the Scribus startup phase, if a master lock can be acquired, the instance +will also sanitize the cache. This includes operations like removing any +orphaned files or fixing reference counters. + + +\section ic_cacheimage Keeping the Cache Image up-to-date + +The cache image is the cache manager's internal representation of all files +in the on-disk cache. It is a tree of ScImageCacheDir and ScImageCacheFile +objects. The ScImageCacheDir objects emit signals when files in the cache +are updated. These signals drive additional operations in the cache manager +like updating the total cache size or the meta age list that keeps track of +the oldest meta files in the cache. + +Each time a Scribus instance performs a write to the cache, it attempts to +acquire a master lock in order to remove old files if necessary. Other +instances might also have modified the cache in between, so it is mandatory +to update the cache image before. + +However, instead of rescanning the whole cache structure, the cache manager +only looks for changes to the \c access files. If a change has been detected +in one directory, its subdirectories are checked recursively. Only directories +that have been modified by other running instances of Scribus need to be +rescanned. So, in the most common case of only one Scribus instance running +at a time, no rescans have to be performed. + +There is one case, however, where the cache image is not kept up-to-date. +Whenever a read-only access to the cache is performed, the corresponding +meta file is touched to prevent it from being deleted when the cache is +cleaned up. This operation does not directly trigger an update of the +cache image. Updating the modification timestamp is delayed until a cache +cleanup becomes necessary. Before the oldest metafile is actually deleted +from the cache, its timestamp is checked and it will only be removed if +it is still the oldest file in the cache. Otherwise, it's position in the +MetaAgeList will be updated. The main driver behind this is that cache +reads should be cheap and not require any locking. However, any changes +to the cache need to be reflected in the \c access files, which would in +turn require locking. + + +\section ic_accessing Accessing the Image Cache + +To access the image cache, a ScImageCacheProxy object is needed. It provides +all necessary functionality to read and write images in the cache. See +pageitem.cpp and scimage.cpp for examples. + +Internally, write accesses to the cache are bracketed with the help of an +ScImageCacheWriteAction object. This object is being notified of all files +that participate in the cache access and will carry out all necessary locking, +updating of the \c access files and notifying the cache manager of any changes. + + +\section ic_performance Performance Measurements + +The following table shows wallclock and real CPU times for loading different +documents in Scribus. In most cases, documents have been loaded multiple +times. Before the first load, the filesystem cache was completely flushed. + +As can be seen, there is no difference in load times if the cache is disabled +in the Scribus preferences. This is important for users who wish to disable +the cache (for whatever reasons). + +Also, first load times are not severely longer if the cache is enabled. In the +worst case, the first load time was less than 20% longer. Most of that time is +spent compressing and writing the cache images, which can be seen in the last +rows of the table. If the images are already found in the cache and only the +meta files have to be created, the load time is almost equivalent to the load +time without cache support. + +However, load times are significantly shorter for the second and third load of +the document. The reason for the third load time being even shorter is that +the cache files are likely to still be present in the filesystem cache. +Usually, re-loading a document is 20 to 50 times faster with the cache enabled. +All measurements were done with medium resolution (72dpi) cache image files and +with the default cache image compression level of 1. Raising the compression +level beyond 4 will mainly slow down the first load of images. Setting it to +zero will significantly increase the cache file size. + +\verbatim +------------------------------------------------------------------------------- + trunk original trunk with cache trunk with cache + no cache support cache disabled cache enabled +------------------------------------------------------------------------------- + wall real wall real wall real +------------------------------------------------------------------------------- + +1 page document +5 small images +on local disc + + 1. load 9.802 2.060 9.661 2.070 9.517 2.610 + 2. load 1.802 1.640 1.812 1.750 0.585 0.530 + 3. load 1.744 1.630 1.709 1.630 0.547 0.470 + +266 page document +2.6 GiB of TIFFs +on local disk +(CMS disabled) + + 1. load 235.080 194.850 233.662 192.800 277.886 226.750 + 2. load 226.178 191.510 225.340 191.920 11.276 9.270 + 3. load 227.191 191.580 7.632 7.320 + +160 page document +2.6 GiB of TIFFs +on network drive +(CMS enabled) + + 1. load 979.028 602.100 1011.878 631.290 + 1. load [1] 985.161 608.530 + 2. load 972.407 599.280 34.296 25.860 + 3. load 18.605 18.310 + +------------------------------------------------------------------------------- + [1] image files already found in cache +------------------------------------------------------------------------------- +\endverbatim + +******************************************************************************/ + +namespace { + const int LOCKFILE_MAX_AGE_SECONDS = 3600; +} + + +ScImageCacheManager::MetaAgeList::MetaAgeList() +{ +} + +void ScImageCacheManager::MetaAgeList::insert(ScImageCacheFile *p) +{ + m_fa.insert(qLowerBound(m_fa.begin(), m_fa.end(), p, ageLessThan), p); +} + +void ScImageCacheManager::MetaAgeList::update(ScImageCacheFile *p, const QFileInfo & newInfo) +{ + remove(p); + p->update(newInfo); + insert(p); +} + +void ScImageCacheManager::MetaAgeList::remove(ScImageCacheFile *p) +{ + FAL::iterator i = qBinaryFind(m_fa.begin(), m_fa.end(), p, ageLessThan); + Q_ASSERT(i != m_fa.end()); + if (i != m_fa.end()) + m_fa.erase(i); + else + scDebug() << "BUG:" << p->path() << "not found in meta age list"; +} + +bool ScImageCacheManager::MetaAgeList::ageLessThan(const ScImageCacheFile *a, const ScImageCacheFile *b) +{ + return a->modified() < b->modified() || (a->modified() == b->modified() && a < b); +} + +ScImageCacheFile *ScImageCacheManager::MetaAgeList::getOldest() +{ + return m_fa.isEmpty() ? 0 : m_fa.front(); +} + + + +ScImageCacheManager & ScImageCacheManager::instance() +{ + static ScImageCacheManager instance; + return instance; +} + +ScImageCacheManager::ScImageCacheManager() + : m_isEnabled(false), m_haveMasterLock(false), m_inCleanup(false), m_writeLockCount(0), + m_compressionLevel(-1), m_maxEntries(0), m_maxSizeMiB(0), m_maxTotalSize(0), + m_totalCacheSize(0), m_writeLockFile(0), m_root(0) +{ +} + +ScImageCacheManager::~ScImageCacheManager() +{ + // no tryCleanup here, I guess we rather want Scribus to exit fast + if (m_root) + delete m_root; +} + +QString ScImageCacheManager::absolutePath(const QString & fn) +{ + QString rv(ScPaths::getImageCacheDir() + fn); + while (rv.endsWith('/')) + rv.chop(1); + return rv; +} + +void ScImageCacheManager::cleanupLockDir() +{ + scDebug() << "cleaning up lock files"; + + QDirIterator di(lockDir()); + QDateTime now = QDateTime::currentDateTime(); + + QFileInfo masterInfo(masterLockFile()); + bool masterLockFound = false; + + while (di.hasNext()) + { + di.next(); + + QFileInfo info = di.fileInfo(); + + if (info.suffix() == ScLockedFile::lockSuffix) + { + int age = info.lastModified().secsTo(now); + if (age > LOCKFILE_MAX_AGE_SECONDS) + { + scDebug() << "removing old cache lock file" << info.filePath() << "(age =" << age << "seconds)"; + if (info.isFile()) + { + if (!QFile::remove(info.filePath())) + scDebug() << "could not remove file" << info.filePath(); + } + else if (info.isDir()) + { + if (info == masterInfo) + masterLockFound = true; + } + } + } + } + + if (masterLockFound) + { + QString masterLockRemove = masterLockFile() + ".remove"; + QDir masterRemove(masterLockRemove); + + scDebug() << "safely removing" << masterLockFile(); + + if (!masterRemove.mkdir(masterLockRemove)) + { + scDebug() << "failed to create" << masterLockRemove; + return; + } + + // At this point, no other instance will attempt to delete the + // master lock *except* for an instance that has just acquired + // the lock after removing the stale lock before. So we check + // again whether the master lock is still present and old + // enough for us to safely remove it. + + // Should, for whatever reason, Scribus crash before the end of + // this routine, the cache will be unusable unless someone cleans + // up the lock directory manually. + + masterInfo.refresh(); + + if (masterInfo.exists()) + { + if (masterInfo.lastModified().secsTo(now) > LOCKFILE_MAX_AGE_SECONDS) + { + if (!masterRemove.rmdir(masterLockFile())) + scDebug() << "could not remove directory" << masterLockFile(); + } + else + scDebug() << masterLockFile() << "has been reincarnated"; + } + else + scDebug() << masterLockFile() << "is already gone"; + + if (!masterRemove.rmdir(masterLockRemove)) + scDebug() << "failed to remove" << masterLockRemove; + } +} + +void ScImageCacheManager::removeMasterLock() +{ + extern bool emergencyActivated; + Q_ASSERT(emergencyActivated); + if (!emergencyActivated) + { + scDebug() << "NEVER call removeMasterLock() when not in an emergency"; + return; + } + if (!m_haveMasterLock) + { + scDebug() << "no master lock set by this instance"; + return; + } + scDebug() << "removing master lock file"; + QDir master(masterLockFile()); + if (!master.rmdir(masterLockFile())) + { + scDebug() << "failed to remove master lock directory"; + return; + } + m_haveMasterLock = false; +} + +void ScImageCacheManager::sanitizeCache() +{ + scDebug() << "sanitizing image cache"; + + QFileInfo masterInfo(masterLockFile()); + QDirIterator di(ScPaths::getImageCacheDir(), QDirIterator::Subdirectories); + QDir dir(ScPaths::getImageCacheDir()); + + QHash<QString, QString> metafile; // meta-filename => base + QHash<QString, int> reffile; // ref-filename => refcount + QHash<QString, int> imgfile; // img-filename => 0 + + ScImageCacheWriteAction action(true); + action.start(); + + while (di.hasNext()) + { + di.next(); + + QFileInfo info = di.fileInfo(); + QString relFile = dir.relativeFilePath(info.filePath()); + + if (info.suffix() == ScLockedFile::lockSuffix) + { + // any lock files outside the lock directory must be leftovers, + // regardless of age, as we've acquired the master lock + + if (info != masterInfo) + { + scDebug() << "removing stale lock file" << relFile; + if (dir.rmdir(info.filePath())) + action.add(relFile); + else + scDebug() << "could not remove" << info.filePath(); + } + } + else if (info.isFile()) + { + if (info.suffix() == ScImageCacheProxy::metaSuffix) + { + QString base = ScImageCacheProxy::getBaseName(relFile); + if (base.isEmpty()) + { + scDebug() << "removing invalid meta file" << relFile; + if (QFile::remove(info.filePath())) + action.add(relFile); + else + scDebug() << "could not remove" << info.filePath(); + } + else + metafile[relFile] = base; + } + else if (info.suffix() == ScImageCacheProxy::referenceSuffix) + { + int refcount; + if (!ScImageCacheProxy::getRefCount(relFile, refcount)) + { + scDebug() << "removing invalid reference file" << relFile; + if (QFile::remove(info.filePath())) + action.add(relFile); + else + scDebug() << "could not remove" << info.filePath(); + } + else + reffile[relFile] = refcount; + } + else if (info.suffix() == ScImageCacheProxy::imageSuffix) + imgfile[relFile] = 0; + else if (di.fileName() != ScImageCacheDir::accessFileName) + scDebug() << "unknown file in cache" << di.fileName(); + } + } + + QRegExp reImg(ScImageCacheProxy::imageSuffix + "$"); + QRegExp reRef(ScImageCacheProxy::referenceSuffix + "$"); + + QHash<QString, int>::iterator isi; + + // delete all image files without reference file + + isi = imgfile.begin(); + while (isi != imgfile.end()) + { + QString ref = isi.key(); + ref.replace(reImg, ScImageCacheProxy::referenceSuffix); + if (!reffile.contains(ref)) + { + scDebug() << "removing image file without reference" << isi.key(); + if (QFile::remove(absolutePath(isi.key()))) + action.add(isi.key()); + else + scDebug() << "could not remove" << absolutePath(isi.key()); + isi = imgfile.erase(isi); + } + else + isi++; + } + + // delete all reference files without image file + + isi = reffile.begin(); + while (isi != reffile.end()) + { + QString img = isi.key(); + img.replace(reRef, ScImageCacheProxy::imageSuffix); + if (!imgfile.contains(img)) + { + scDebug() << "removing reference file without image" << isi.key(); + if (QFile::remove(absolutePath(isi.key()))) + action.add(isi.key()); + else + scDebug() << "could not remove" << absolutePath(isi.key()); + isi = reffile.erase(isi); + } + else + isi++; + } + + // find all metafiles that don't reference existing reference files + // these can be directly deleted + + QHash<QString, QString>::iterator iss; + QHash<QString, int> references; // ref-filename => number of references + + iss = metafile.begin(); + while (iss != metafile.end()) + { + QString ref = *iss + "." + ScImageCacheProxy::referenceSuffix; + if (!reffile.contains(ref)) + { + scDebug() << "removing orphaned meta file" << iss.key(); + if (QFile::remove(absolutePath(iss.key()))) + action.add(iss.key()); + else + scDebug() << "could not remove" << iss.key(); + iss = metafile.erase(iss); + } + else + { + QFileInfo info(absolutePath(iss.key())); + if (!references.contains(ref)) + references[ref] = 1; + else + references[ref]++; + iss++; + } + } + + // find all reference files that are not referenced by any metafile + // these, and their corresponding image files, can be directly deleted + // fix reference counts for remaining reference files + + isi = reffile.begin(); + while (isi != reffile.end()) + { + int newRefCount; + + if (!references.contains(isi.key())) + { + QString ref = isi.key(); + QString img = ref; + img.replace(reRef, ScImageCacheProxy::imageSuffix); + scDebug() << "removing orphaned reference/image files" << ref << img; + if (QFile::remove(absolutePath(ref))) + action.add(ref); + else + scDebug() << "could not remove" << absolutePath(ref); + if (QFile::remove(absolutePath(img))) + action.add(img); + else + scDebug() << "could not remove" << absolutePath(img); + } + else if (*isi != (newRefCount = references[isi.key()])) + { + scDebug() << "fixing reference count for" << isi.key() << "old =" << *isi << "new =" << newRefCount; + if (ScImageCacheProxy::fixRefCount(isi.key(), newRefCount)) + action.add(isi.key()); + else + scDebug() << "could not fix reference count for" << isi.key(); + } + isi++; + } + + action.commit(); + + scDebug() << "finished sanitizing image cache"; +} + +void ScImageCacheManager::tryCleanup() +{ + if (m_inCleanup) + return; + + scDebug() << "attempting to acquire master lock"; + + if (!acquireMasterLock()) + return; + + cleanupCache(); + + scDebug() << "releasing master lock"; + + releaseMasterLock(); +} + +void ScImageCacheManager::updateCache() +{ + scDebug() << "updating cache"; + + if (!m_root) + { + QStringList suffixes; + suffixes << ScImageCacheProxy::metaSuffix << ScImageCacheProxy::referenceSuffix << ScImageCacheProxy::imageSuffix; + + m_root = new ScImageCacheDir(ScPaths::getImageCacheDir()); + Q_CHECK_PTR(m_root); + + if (!m_root) + return; + + for (int j = 0; j < 16; j++) + { + ScImageCacheDir *d1 = m_root->newSubDir(QString::number(j, 16)); + + if (!d1) + { + delete m_root; + m_root = 0; + return; + } + + for (int k = 0; k < 16; k++) + { + ScImageCacheDir *d2 = d1->newSubDir(QString::number(k, 16), true, suffixes); + + if (!d2) + { + delete m_root; + m_root = 0; + return; + } + + connect(d2, SIGNAL(fileCreated(ScImageCacheFile *, const QFileInfo &)), SLOT(fileCreated(ScImageCacheFile *, const QFileInfo &))); + connect(d2, SIGNAL(fileChanged(ScImageCacheFile *, const QFileInfo &)), SLOT(fileChanged(ScImageCacheFile *, const QFileInfo &))); + connect(d2, SIGNAL(fileRemoved(ScImageCacheFile *)), SLOT(fileRemoved(ScImageCacheFile *))); + } + } + } + + m_root->update(); +} + +void ScImageCacheManager::cleanupCache() +{ + m_inCleanup = true; + + updateCache(); + + // remove oldest entries until limits are reached + + scDebug() << "removing old cache entries"; + + scDebug() << "total size:" << m_totalCacheSize << "/ max:" << m_maxTotalSize; + scDebug() << "meta count:" << m_metaAge.count() << "/ max:" << m_maxEntries; + + while (m_totalCacheSize > m_maxTotalSize || m_metaAge.count() > m_maxEntries) + { + ScImageCacheFile *p = getOldestCacheEntry(); + if (!ScImageCacheProxy::removeCacheEntry(p->path(true), true)) + break; + scDebug() << "total size:" << m_totalCacheSize << "/ max:" << m_maxTotalSize; + scDebug() << "meta count:" << m_metaAge.count() << "/ max:" << m_maxEntries; + } + + m_inCleanup = false; +} + +void ScImageCacheManager::initialize() +{ + scDebug() << "starting cache manager initialization"; + + // no need to have a lock here, as we just create the basic cache structure + + if (enabled()) + { + cleanupLockDir(); + + scDebug() << "attempting to acquire master lock"; + + if (acquireMasterLock()) + { + sanitizeCache(); + cleanupCache(); + + scDebug() << "releasing master lock"; + + releaseMasterLock(); + } + } + + scDebug() << "cache manager initialization finished"; +} + +void ScImageCacheManager::fileCreated(ScImageCacheFile * file, const QFileInfo & info) +{ + QString relFile = file->path(true); + scDebug() << "created" << relFile; + m_totalCacheSize += file->size(); + if (info.suffix() == ScImageCacheProxy::metaSuffix) + m_metaAge.insert(file); +} + +void ScImageCacheManager::fileChanged(ScImageCacheFile * file, const QFileInfo & info) +{ + QString relFile = file->path(true); + scDebug() << "updated" << relFile; + m_totalCacheSize -= file->size(); + if (info.suffix() == ScImageCacheProxy::metaSuffix) + m_metaAge.update(file, info); + else + file->update(info); + m_totalCacheSize += file->size(); +} + +void ScImageCacheManager::fileRemoved(ScImageCacheFile * file) +{ + QString relFile = file->path(true); + scDebug() << "removed" << relFile; + m_totalCacheSize -= file->size(); + if (relFile.section('.', -1) == ScImageCacheProxy::metaSuffix) + m_metaAge.remove(file); +} + +ScImageCacheFile *ScImageCacheManager::getOldestCacheEntry() +{ + ScImageCacheFile *file; + + while ((file = m_metaAge.getOldest()) != 0) + { + QFileInfo info(file->path()); + if (!info.exists()) + scDebug() << "BUG: oldest file" << file->path() << "already removed?"; + else if (info.lastModified() == file->modified()) + break; + m_root->updateFile(file->path(true)); + } + + return file; +} +void ScImageCacheManager::setEnabled(bool enableCache) +{ + m_isEnabled = enableCache; +} + +bool ScImageCacheManager::setMaxCacheSizeMiB(int maxCacheSizeMiB) +{ + if (maxCacheSizeMiB < 1) + return false; + m_maxSizeMiB = maxCacheSizeMiB; + m_maxTotalSize = Q_INT64_C(1048576)*static_cast<qint64>(maxCacheSizeMiB); + return true; +} + +bool ScImageCacheManager::setMaxCacheEntries(int maxCacheEntries) +{ + if (maxCacheEntries < 1) + return false; + m_maxEntries = maxCacheEntries; + return true; +} + +bool ScImageCacheManager::setCompressionLevel(int level) +{ + if (-1 <= m_compressionLevel && m_compressionLevel <= 9) + { + m_compressionLevel = level; + return true; + } + return false; +} + +int ScImageCacheManager::compressionLevel() const +{ + return m_compressionLevel; +} + +QString ScImageCacheManager::lockDir() +{ + return ScPaths::getImageCacheDir() + "locks/"; +} + +QString ScImageCacheManager::masterLockFile() +{ + return lockDir() + "master." + ScLockedFile::lockSuffix; +} + +QString ScImageCacheManager::writeLockTemplate() +{ + return lockDir() + "XXXXXXXX." + ScLockedFile::lockSuffix; +} + +bool ScImageCacheManager::createLockDir() +{ + QDir dir(lockDir()); + return dir.exists() || dir.mkpath(lockDir()); +} + +bool ScImageCacheManager::acquireMasterLock() +{ + Q_ASSERT(!m_haveMasterLock); + Q_ASSERT(m_writeLockCount == 0); + if (m_haveMasterLock) + { + scDebug() << "BUG: master lock already acquired"; + return false; + } + if (m_writeLockCount > 0) + { + scDebug() << "BUG: attempt to acquire master lock with active write locks"; + return false; + } + if (!createLockDir()) + { + scDebug() << "failed to create lock directory"; + return false; + } + QDir master(masterLockFile()); + if (master.exists()) + { + scDebug() << "master lock already active"; + return false; + } + if (!master.mkdir(masterLockFile())) + { + scDebug() << "failed to create master lock directory"; + return false; + } + QDirIterator di(lockDir()); + while (di.hasNext()) + { + di.next(); + QFileInfo info = di.fileInfo(); + if (info.isFile() && info.suffix() == ScLockedFile::lockSuffix) + { + scDebug() << "lock file present, cannot acquire master lock"; + if (!master.rmdir(masterLockFile())) + scDebug() << "failed to remove master lock directory"; + return false; + } + } + m_haveMasterLock = true; + return true; +} + +bool ScImageCacheManager::releaseMasterLock() +{ + Q_ASSERT(m_haveMasterLock); + Q_ASSERT(m_writeLockCount == 0); + if (!m_haveMasterLock) + { + scDebug() << "BUG: master lock not acquired"; + return false; + } + if (m_writeLockCount > 0) + { + scDebug() << "BUG: release of master lock with active write locks"; + return false; + } + QDir master(masterLockFile()); + if (!master.rmdir(masterLockFile())) + { + scDebug() << "failed to remove master lock directory"; + return false; + } + m_haveMasterLock = false; + return true; +} + +bool ScImageCacheManager::acquireWriteLock() +{ + if (!m_haveMasterLock && m_writeLockCount == 0) + { + Q_ASSERT(m_writeLockFile == 0); + if (!createLockDir()) + { + scDebug() << "failed to create lock directory"; + return false; + } + QDir master(masterLockFile()); + if (master.exists()) + { + scDebug() << "master lock active"; + return false; + } + m_writeLockFile = new QTemporaryFile(writeLockTemplate()); + Q_CHECK_PTR(m_writeLockFile); + if (!m_writeLockFile) + return false; + if (!m_writeLockFile->open()) + { + scDebug() << "failed to create write lock file"; + delete m_writeLockFile; + m_writeLockFile = 0; + return false; + } + if (master.exists()) + { + scDebug() << "master lock active"; + delete m_writeLockFile; + m_writeLockFile = 0; + return false; + } + } + m_writeLockCount++; + return true; +} + +bool ScImageCacheManager::releaseWriteLock() +{ + if (m_writeLockCount == 0) + { + Q_ASSERT(m_writeLockFile == 0); + return false; + } + m_writeLockCount--; + if (!m_haveMasterLock) + { + Q_ASSERT(m_writeLockFile != 0); + if (m_writeLockCount == 0) + { + delete m_writeLockFile; + m_writeLockFile = 0; + } + } + return true; +} + +bool ScImageCacheManager::updateAccess(const QString & dir, AccessCounter from, AccessCounter to) +{ + // don't propagate updates until we have scanned the cache at least once + return m_root ? m_root->updateAccess(dir, from, to) : false; +} + +bool ScImageCacheManager::updateFile(const QString & file) +{ + // don't propagate updates until we have scanned the cache at least once + return m_root ? m_root->updateFile(file) : false; +} + Index: scribus/scimagecacheproxy.h =================================================================== --- scribus/scimagecacheproxy.h (revision 0) +++ scribus/scimagecacheproxy.h (revision 0) @@ -0,0 +1,171 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#ifndef SCIMAGECACHEPROXY_H +#define SCIMAGECACHEPROXY_H + +#include "scconfig.h" +#include "scribusapi.h" + +#include <QImage> +#include <QString> +#include <QMap> + +class ScImage; +class ScLockedFile; +class ScImageCacheManager; + +/** + * @brief Scribus image cache proxy + * @author Marcus Holland-Moritz + */ +class SCRIBUS_API ScImageCacheProxy +{ +public: + static const QString metaSuffix; //!< Meta file suffix + static const QString referenceSuffix; //!< Reference file suffix + static const QString imageSuffix; //!< Cache image file suffix + + /** + * @brief Construct a cache proxy object + * @param fn Full path to the original image file + */ + ScImageCacheProxy(const QString & fn); + ~ScImageCacheProxy(); + + /** + * @brief Check if the image cache is enabled + */ + bool enabled() const { return m_isEnabled; } + /** + * @brief Get original image file name + */ + const QString & getFilename() const { return m_filename; } + /** + * @brief Load image from cache + * @param image QImage object to which to load the cached image + * @return \c true if the image could be loaded, \c false otherwise + */ + bool load(QImage & image); + /** + * @brief Save image to cache + * @param image QImage object from which to save the cached image + * @return \c true if the image could be saved, \c false otherwise + */ + bool save(const QImage & image); + /** + * @brief Touch an image in the cache + * @return \c true if the image could be touched, \c false otherwise + */ + bool touch() const; + /** + * @brief Add or overwrite image metadata + */ + void addMetadata(const QString & key, const QString & value); + /** + * @brief Add or overwrite an image modifier + */ + void addModifier(const QString & key, const QString & value); + /** + * @brief Delete an image modifier + */ + void delModifier(const QString & key); + /** + * @brief Check if the cached image can be used in place of the original + * @return \c true if the cached image can be used, \c false otherwise + */ + bool canUseCachedImage() const; + + /** + * @brief Add image information + */ + void addInfo(const QString & key, const QString & value); + /** + * @brief Retrieve image information + */ + QString getInfo(const QString & key) const; + + /** + * @brief Get base name of reference/image file from meta file + * @param metafile Path to meta file relative to the cache root directory + * @return Reference/image file basename, i.e. path and filename without the suffix + */ + static QString getBaseName(const QString & metafile); + /** + * @brief Get reference count from a reference file + * @param reffile Path to reference file relative to the cache root directory + * @param refcount Reference to the variable receiving the reference counter + * @return \c true if the reference count could be read, \c false otherwise + */ + static bool getRefCount(const QString & reffile, int & refcount); + /** + * @brief Set reference count in a reference file + * @param reffile Path to reference file relative to the cache root directory + * @param refcount The reference count to write to the file + * @return \c true if the reference count could be written, \c false otherwise + */ + static bool fixRefCount(const QString & reffile, int refcount); + /** + * @brief Remove an entry from the image cache + * + * Attempts to remove a meta file from the image cache. Decrements the + * reference counter of the referenced image. If the reference counter + * drops to zero, the image and its reference file will also be removed. + * + * @param metafile Path to meta file relative to the cache root directory + * @param haveMasterLock \c true if a master lock has already been acquired + * @return \c true if the reference count could be written, \c false otherwise + */ + static bool removeCacheEntry(const QString & metafile, bool haveMasterLock = false); + +private: + // Don't turn this into a QHash, element order is important + typedef QMap<QString, QString> MetaMap; + + const QString m_filename; + const bool m_isEnabled; + mutable QString m_metanameCache; + MetaMap m_metadata; + MetaMap m_modifier; + MetaMap m_imginfo; + + static QString imageFile(const QString & base); + static QString referenceFile(const QString & base); + + static bool createCacheDir(); + static QString addDirLevels(QString name); + + const QString & metaName() const; + QString imageBaseName(const QImage & image) const; + + bool loadMetadata(MetaMap *meta, MetaMap *mod, MetaMap *info, QString *base) const; + + static bool loadMetadata(ScLockedFile *file, MetaMap *meta, MetaMap *mod, MetaMap *info, QString *base); + static bool loadMetadata(const QString & fn, MetaMap *meta, MetaMap *mod, MetaMap *info, QString *base); + static void saveMetadata(ScLockedFile *file, const MetaMap & map, const MetaMap & mod, const MetaMap & info, const QString & base); + + static bool getRefCountAbs(const QString & reffile, int & refcount); + static bool loadRef(ScLockedFile *file, int & refcount); + static void saveRef(ScLockedFile *file, int refcount); + static bool refImage(ScLockedFile *file); + static bool unrefImage(ScLockedFile *file, const QString & imageName); +}; + +#endif Index: scribus/scimagecachewriteaction.cpp =================================================================== --- scribus/scimagecachewriteaction.cpp (revision 0) +++ scribus/scimagecachewriteaction.cpp (revision 0) @@ -0,0 +1,182 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#include "sclockedfile.h" +#include "scimagecachemanager.h" +#include "scimagecachewriteaction.h" + +#define SC_DEBUG_FILE defined(DEBUG_SCIMAGECACHE) +#include "scdebug.h" + +ScImageCacheWriteAction::ScImageCacheWriteAction(bool haveMasterLock) + : m_locked(false), m_haveMasterLock(haveMasterLock) +{ +} + +ScImageCacheWriteAction::~ScImageCacheWriteAction() +{ + if (m_locked) + unlock(); // don't commit! + clear(); +} + +void ScImageCacheWriteAction::clear() +{ + for (FileMap::iterator i = m_access.begin(); i != m_access.end(); i++) + delete *i; + m_access.clear(); +} + +bool ScImageCacheWriteAction::start() +{ + Q_ASSERT(!m_locked); + if (m_locked) + { + scDebug() << "BUG: attempt to start action twice"; + return false; + } + if (!ScImageCacheManager::instance().acquireWriteLock()) + { + scDebug() << "failed to acquire cache write lock"; + return false; + } + m_locked = true; + return true; +} + +bool ScImageCacheWriteAction::unlock() +{ + bool rv = true; + if (!m_haveMasterLock) + { + for (FileMap::iterator i = m_access.begin(); i != m_access.end(); i++) + { + ScLockedFile *p = *i; + if (!p->unlock()) + rv = false; + } + } + if (!ScImageCacheManager::instance().releaseWriteLock()) + { + scDebug() << "failed to release cache write lock"; + rv = false; + } + m_locked = false; + return rv; +} + +bool ScImageCacheWriteAction::update(const QString & dir, ScLockedFile *p, AccessCounter & from, AccessCounter & to) +{ + AccessCounter counter = 0; + if (p->exists() && !ScImageCacheDir::lastAccess(dir, counter)) + { + scDebug() << "failed to read" << p->name() << "dir =" << dir; + return false; + } + from = counter; + counter++; + if (!p->open()) + { + scDebug() << "failed to open" << p->name() << "for writing"; + return false; + } + QTextStream out(p->io()); + out << counter; + to = counter; + return true; +} + +bool ScImageCacheWriteAction::commit() +{ + bool rv = true; + Q_ASSERT(m_locked); + if (!m_locked) + { + scDebug() << "BUG: attempt to release non-locked cache write lock"; + return false; + } + ScImageCacheManager & scm = ScImageCacheManager::instance(); + foreach (QString file, m_files) + scm.updateFile(file); + for (FileMap::iterator i = m_access.begin(); i != m_access.end(); i++) + { + ScLockedFile *p = *i; + AccessCounter from, to; + if (update(i.key(), p, from, to)) + { + if (p->commit()) + { + scDebug() << "updated" << p->name() << "from" << from << "to" << to; + scm.updateAccess(i.key(), from, to); + } + else + { + scDebug() << "failed to commit changes to" << p->name(); + rv = false; + } + } + else + rv = false; + } + if (!unlock()) + rv = false; + clear(); + if (!m_haveMasterLock) + scm.tryCleanup(); + return rv; +} + +bool ScImageCacheWriteAction::add(const QString & file) +{ + Q_ASSERT(m_locked); + if (!m_locked) + { + scDebug() << "BUG: attempt to add" << file << "to action without start"; + return false; + } + + QStringList dirs = file.split('/'); + QStringList dl; + + while (dirs.count() > 1) + { + dirs.removeLast(); + QString d = dirs.join("/"); + if (!m_access.contains(d)) + dl << d; + } + + while (!dl.isEmpty()) + { + QString d = dl.takeLast(); + ScLockedFile *p = new ScLockedFileRW(ScImageCacheManager::absolutePath(d) + "/" + ScImageCacheDir::accessFileName); + Q_CHECK_PTR(p); + if (!p) + return false; + if (!m_haveMasterLock && !p->lock()) + return false; + m_access[d] = p; + } + + m_files << file; + + return true; +} + Index: scribus/sclockedfile.cpp =================================================================== --- scribus/sclockedfile.cpp (revision 0) +++ scribus/sclockedfile.cpp (revision 0) @@ -0,0 +1,214 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#include <QDir> +#include <QFileInfo> + +#include "sclockedfile.h" + +#define SC_DEBUG_FILE defined(DEBUG_SCLOCKEDFILE) +#include "scdebug.h" + +const QString ScLockedFile::lockSuffix("lock"); + +ScLockedFile::ScLockedFile() + : m_fileName(), m_isOpened(false), m_isLocked(false) +{ +} + +ScLockedFile::ScLockedFile(const QString & name) + : m_fileName(name), m_isOpened(false), m_isLocked(false) +{ +} + +ScLockedFile::~ScLockedFile() +{ + if (m_isLocked) + unlock(); +} + +void ScLockedFile::setName(const QString & name) +{ + m_fileName = name; +} + +QString ScLockedFile::lockName() const +{ + return m_fileName + "." + lockSuffix; +} + +bool ScLockedFile::lock() +{ + Q_ASSERT(!m_isLocked); + if (m_isLocked) + { + scDebug() << "BUG:" << m_fileName << "is already locked"; + return false; + } + QDir cdir; + m_isLocked = cdir.mkdir(lockName()); + if (!m_isLocked) + scDebug() << "cannot lock" << m_fileName; + return m_isLocked; +} + +bool ScLockedFile::unlock() +{ + Q_ASSERT(m_isLocked); + if (!m_isLocked) + { + scDebug() << "BUG:" << m_fileName << "is not locked"; + return false; + } + QDir cdir; + m_isLocked = !cdir.rmdir(lockName()); + if (m_isLocked) + scDebug() << "cannot unlock" << m_fileName; + return !m_isLocked; +} + +bool ScLockedFile::exists() const +{ + return QFile::exists(m_fileName); +} + +bool ScLockedFile::remove() const +{ + return QFile::remove(m_fileName); +} + +bool ScLockedFile::createPath() const +{ + QFileInfo info(m_fileName); + QString path = info.path(); + QDir dir(path); + if (dir.exists()) + return true; + scDebug() << "creating directory" << path; + return dir.mkpath(path); +} + + +ScLockedFileRW::ScLockedFileRW() +{ +} + +ScLockedFileRW::ScLockedFileRW(const QString & name) + : ScLockedFile(name) +{ +} + +QString ScLockedFileRW::templateName(const QFileInfo & info) +{ + return info.dir().path() + "/" + info.baseName() + "_XXXXXX." + info.completeSuffix(); +} + +bool ScLockedFileRW::open() +{ + Q_ASSERT(!m_fileName.isEmpty()); + + if (m_fileName.isEmpty()) + { + scDebug() << "BUG: file name not set"; + return false; + } + + QFileInfo info(m_fileName); + if (info.exists() && !info.isWritable()) + { + scDebug() << "final file is not writeable, aborting open for" << m_fileName; + return false; + } + + m_file.setFileTemplate(templateName(info)); + m_isOpened = m_file.open(); + + if (!m_isOpened) + scDebug() << "could not open read/write" << m_fileName; + + return m_isOpened; +} + +bool ScLockedFileRW::commit() +{ + Q_ASSERT(m_isOpened); + + if (!m_isOpened) + { + scDebug() << "BUG: cannot commit," << m_fileName << "is not opened"; + return false; + } + + if (exists() && !remove()) + { + scDebug() << "cannot commit," << m_fileName << "cannot be removed"; + return false; + } + + m_isOpened = false; + + QString tmpName = m_file.fileName(); + + if (!m_file.rename(m_fileName)) + { + scDebug() << "cannot commit," << tmpName << "cannot be renamed to" << m_fileName; + return false; + } + + m_file.setAutoRemove(false); + + return true; +} + + +ScLockedFileRO::ScLockedFileRO() +{ +} + +ScLockedFileRO::ScLockedFileRO(const QString & name) + : ScLockedFile(name) +{ +} + +bool ScLockedFileRO::open() +{ + Q_ASSERT(!m_fileName.isEmpty()); + + if (m_fileName.isEmpty()) + { + scDebug() << "BUG: file name not set"; + return false; + } + + m_file.setFileName(m_fileName); + m_isOpened = m_file.open(QIODevice::ReadOnly); + + if (!m_isOpened && exists()) + scDebug() << "could not open read-only" << m_fileName; + + return m_isOpened; +} + +bool ScLockedFileRO::commit() +{ + Q_ASSERT(false); + scDebug() << "BUG: cannot commit," << m_fileName << "is read-only"; + return false; +} Index: scribus/scimagecachefile.cpp =================================================================== --- scribus/scimagecachefile.cpp (revision 0) +++ scribus/scimagecachefile.cpp (revision 0) @@ -0,0 +1,82 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#include <QFileInfo> + +#include "scimagecachedir.h" +#include "scimagecachefile.h" + +#define SC_DEBUG_FILE defined(DEBUG_SCIMAGECACHE) +#include "scdebug.h" + +ScImageCacheFile::ScImageCacheFile(const QString & name, ScImageCacheDir *parent) + :m_name(name), m_parent(parent) +{ + QFileInfo info(path()); + m_modified = info.lastModified(); + m_size = info.size(); +} + +ScImageCacheFile::~ScImageCacheFile() +{ +} + +QString ScImageCacheFile::path(bool relative) const +{ + return m_parent ? m_parent->path(relative) + "/" + m_name : m_name; +} + +qint64 ScImageCacheFile::size() const +{ + return m_size; +} + +bool ScImageCacheFile::hasChanged(const QFileInfo & info) const +{ + return info.lastModified() != m_modified || info.size() != m_size; +} + +bool ScImageCacheFile::hasChanged() const +{ + QFileInfo info(path()); + return hasChanged(info); +} + +bool ScImageCacheFile::update(const QFileInfo & info) +{ + bool changed = false; + if (info.lastModified() != m_modified) + { + m_modified = info.lastModified(); + changed = true; + } + if (info.size() != m_size) + { + m_size = info.size(); + changed = true; + } + return changed; +} + +bool ScImageCacheFile::update() +{ + QFileInfo info(path()); + return update(info); +} Index: scribus/main_win32.cpp =================================================================== --- scribus/main_win32.cpp (revision 14532) +++ scribus/main_win32.cpp (working copy) @@ -46,6 +46,7 @@ #include "scribusapp.h" #include "scribuscore.h" #include "scribus.h" +#include "scimagecachemanager.h" #include "scconfig.h" @@ -289,6 +290,7 @@ ScMW->emergencySave(); ScMW->close(); } + ScImageCacheManager::instance().removeMasterLock(); } ExitProcess(255); } Index: scribus/prefsmanager.cpp =================================================================== --- scribus/prefsmanager.cpp (revision 14532) +++ scribus/prefsmanager.cpp (working copy) @@ -492,6 +492,10 @@ appPrefs.pdfPrefs.fitWindow = false; appPrefs.pdfPrefs.PageLayout = PDFOptions::SinglePage; appPrefs.pdfPrefs.openAction = ""; + appPrefs.imageCachePrefs.cacheEnabled = false; + appPrefs.imageCachePrefs.maxCacheSizeMiB = 1000; + appPrefs.imageCachePrefs.maxCacheEntries = 1000; + appPrefs.imageCachePrefs.compressionLevel = 1; //Attribute setup appPrefs.itemAttrPrefs.defaultItemAttributes.clear(); @@ -1724,6 +1728,13 @@ liElem.setAttribute("useStandardLI", static_cast<int>(appPrefs.miscPrefs.useStandardLI)); liElem.setAttribute("paragraphsLI", appPrefs.miscPrefs.paragraphsLI); elem.appendChild(liElem); + // image cache + QDomElement icElem = docu.createElement("ImageCache"); + icElem.setAttribute("cacheEnabled", appPrefs.imageCachePrefs.cacheEnabled); + icElem.setAttribute("maxCacheSizeMiB", appPrefs.imageCachePrefs.maxCacheSizeMiB); + icElem.setAttribute("maxCacheEntries", appPrefs.imageCachePrefs.maxCacheEntries); + icElem.setAttribute("compressionLevel", appPrefs.imageCachePrefs.compressionLevel); + elem.appendChild(icElem); // write file bool result = false; QFile f(ho); @@ -2449,6 +2460,14 @@ appPrefs.miscPrefs.useStandardLI = static_cast<bool>(dc.attribute("useStandardLI", "0").toInt()); appPrefs.miscPrefs.paragraphsLI = dc.attribute("paragraphsLI", "10").toInt(); } + // cache manager + if (dc.tagName() == "ImageCache") + { + appPrefs.imageCachePrefs.cacheEnabled = static_cast<bool>(dc.attribute("cacheEnabled", "0").toInt()); + appPrefs.imageCachePrefs.maxCacheSizeMiB = dc.attribute("maxCacheSizeMiB", "1000").toInt(); + appPrefs.imageCachePrefs.maxCacheEntries = dc.attribute("maxCacheEntries", "1000").toInt(); + appPrefs.imageCachePrefs.compressionLevel = dc.attribute("compressionLevel", "1").toInt(); + } DOC=DOC.nextSibling(); } // Some sanity checks Index: scribus/prefsstructs.h =================================================================== --- scribus/prefsstructs.h (revision 14532) +++ scribus/prefsstructs.h (working copy) @@ -352,6 +352,15 @@ { }; +// Image Cache +struct ImageCachePrefs +{ + bool cacheEnabled; //!< Enable the image cache + int maxCacheSizeMiB; //!< Maximum total size of image cache in MiB + int maxCacheEntries; //!< Maximum number of cache entries + int compressionLevel; //!< Cache image compression level (see QImage) +}; + struct ApplicationPrefs { UIPrefs uiPrefs; @@ -379,6 +388,7 @@ CheckerPrefsList checkerPrefsList; StoryEditorPrefs storyEditorPrefs; PrintPreviewPrefs printPreviewPrefs; + ImageCachePrefs imageCachePrefs; QList<ArrowDesc> arrowStyles; QMap<QString, VGradient> defaultGradients; Index: scribus/pageitem.h =================================================================== --- scribus/pageitem.h (revision 14532) +++ scribus/pageitem.h (working copy) @@ -1104,6 +1104,13 @@ void updateConstants(); +private: + /** + * @brief Helper method to create a modifier string from the current image effects list. + * @sa loadImage() + */ + QString getImageEffectsModifier() const; + protected: void drawLockedMarker(ScPainter *p); Index: scribus/scimagestructs.cpp =================================================================== --- scribus/scimagestructs.cpp (revision 14532) +++ scribus/scimagestructs.cpp (working copy) @@ -5,8 +5,42 @@ for which a new license (GPL+exception) is in place. */ +#include "scimagecacheproxy.h" #include "scimagestructs.h" +#include <QByteArray> +#include <QDataStream> + +#define SC_DEBUG_FILE defined(DEBUG_SCIMAGECACHE) +#include "scdebug.h" + +namespace { + const QDataStream::Version dsVersion = QDataStream::Qt_4_0; +} + +const qint32 ExifValues::dsVersion = 1; + +QDataStream & operator<< (QDataStream& stream, const ExifValues & exif) +{ + stream << static_cast<qint32>(exif.width) << static_cast<qint32>(exif.height) << exif.ExposureTime + << exif.ApertureFNumber << static_cast<qint32>(exif.ISOequivalent) << exif.cameraName + << exif.cameraVendor << exif.comment << exif.userComment << exif.artist << exif.copyright + << exif.dateTime << exif.thumbnail; + return stream; +} + +QDataStream & operator>> (QDataStream& stream, ExifValues & exif) +{ + qint32 w, h, iso; + stream >> w >> h >> exif.ExposureTime >> exif.ApertureFNumber >> iso >> exif.cameraName + >> exif.cameraVendor >> exif.comment >> exif.userComment >> exif.artist >> exif.copyright + >> exif.dateTime >> exif.thumbnail; + exif.width = w; + exif.height = h; + exif.ISOequivalent = iso; + return stream; +} + ExifValues::ExifValues(void) { init(); @@ -29,6 +63,8 @@ thumbnail = QImage(); } +const qint32 ImageInfoRecord::iirVersion = 1; + ImageInfoRecord::ImageInfoRecord(void) { init(); @@ -60,3 +96,93 @@ duotoneColors.clear(); exifInfo.init(); } + +bool ImageInfoRecord::canSerialize() const +{ + return PDSpathData.empty() && RequestProps.empty() && layerInfo.empty() && duotoneColors.empty(); +} + +bool ImageInfoRecord::serialize(ScImageCacheProxy & cache) const +{ + if (!canSerialize()) + { + scDebug() << "cannot serialize" << PDSpathData.empty() << RequestProps.empty() << layerInfo.empty() << duotoneColors.empty(); + return false; + } + + cache.addInfo("iirVersion", QString::number(iirVersion)); + cache.addInfo("type", QString::number(static_cast<int>(type))); + cache.addInfo("xres", QString::number(xres)); + cache.addInfo("yres", QString::number(yres)); + cache.addInfo("BBoxX", QString::number(BBoxX)); + cache.addInfo("BBoxH", QString::number(BBoxH)); + cache.addInfo("colorspace", QString::number(static_cast<int>(colorspace))); + cache.addInfo("valid", QString::number(static_cast<int>(valid))); + cache.addInfo("isRequest", QString::number(static_cast<int>(isRequest))); + cache.addInfo("progressive", QString::number(static_cast<int>(progressive))); + cache.addInfo("isEmbedded", QString::number(static_cast<int>(isEmbedded))); + cache.addInfo("lowResType", QString::number(lowResType)); + cache.addInfo("lowResScale", QString::number(lowResScale, 'g', 15)); + cache.addInfo("clipPath", clipPath); + cache.addInfo("profileName", profileName); + + if (exifDataValid) + { + QByteArray exif; + QDataStream es(&exif, QIODevice::WriteOnly); + es.setVersion(dsVersion); + es << ExifValues::dsVersion; + es << exifInfo; + cache.addInfo("exifInfo", exif.toBase64()); + } + + return true; +} + +bool ImageInfoRecord::deserialize(const ScImageCacheProxy & cache) +{ + PDSpathData.clear(); + RequestProps.clear(); + layerInfo.clear(); + duotoneColors.clear(); + usedPath.resize(0); + int v1 = cache.getInfo("iirVersion").toInt(); + if (v1 != iirVersion) + { + scDebug() << "image info version mismatch" << v1 << "!=" << iirVersion; + return false; + } + type = static_cast<ImageTypeEnum>(cache.getInfo("type").toInt()); + xres = cache.getInfo("xres").toInt(); + yres = cache.getInfo("yres").toInt(); + BBoxX = cache.getInfo("BBoxX").toInt(); + BBoxH = cache.getInfo("BBoxH").toInt(); + colorspace = static_cast<ColorSpaceEnum>(cache.getInfo("colorspace").toInt()); + valid = cache.getInfo("valid").toInt() != 0; + isRequest = cache.getInfo("isRequest").toInt() != 0; + progressive = cache.getInfo("progressive").toInt() != 0; + isEmbedded = cache.getInfo("isEmbedded").toInt() != 0; + lowResType = cache.getInfo("lowResType").toInt(); + lowResScale = cache.getInfo("lowResScale").toDouble(); + clipPath = cache.getInfo("clipPath"); + profileName = cache.getInfo("profileName"); + QString exifData = cache.getInfo("exifInfo"); + exifDataValid = !exifData.isNull(); + if (exifDataValid) + { + QByteArray exif = QByteArray::fromBase64(exifData.toAscii()); + QDataStream es(exif); + es.setVersion(dsVersion); + qint32 v2; + es >> v2; + if (v2 != ExifValues::dsVersion) + { + scDebug() << "exif version mismatch" << v2 << "!=" << ExifValues::dsVersion; + return false; + } + es >> exifInfo; + } + + return true; +} + Index: scribus/main_nix.cpp =================================================================== --- scribus/main_nix.cpp (revision 14532) +++ scribus/main_nix.cpp (working copy) @@ -35,6 +35,7 @@ #include "scribusapp.h" #include "scribuscore.h" #include "scribus.h" +#include "scimagecachemanager.h" #include "scconfig.h" @@ -129,6 +130,7 @@ std::cout << sigHdr.toStdString() << std::endl; std::cout << sigLine.toStdString() << std::endl; std::cout << sigMsg.toStdString() << std::endl; + ScImageCacheManager::instance().removeMasterLock(); if (ScribusQApp::useGUI) { ScCore->closeSplash(); Index: scribus/scpaths.h =================================================================== --- scribus/scpaths.h (revision 14532) +++ scribus/scpaths.h (working copy) @@ -63,6 +63,8 @@ /** @brief Return path to application data dir*/ static QString getApplicationDataDir(void); + /** @brief Return path to image cache dir*/ + static QString getImageCacheDir(void); /** @brief Return path to plugin data dir*/ static QString getPluginDataDir(void); /** @brief Return path to user documents*/ Index: scribus/pageitem.cpp =================================================================== --- scribus/pageitem.cpp (revision 14532) +++ scribus/pageitem.cpp (working copy) @@ -49,6 +49,7 @@ #include "resourcecollection.h" #include "scclocale.h" #include "sccolorengine.h" +#include "scimagecacheproxy.h" #include "scpainter.h" #include "scpaths.h" #include "scpattern.h" @@ -5276,6 +5277,24 @@ return transRect.contains(x, y); } +QString PageItem::getImageEffectsModifier() const +{ + bool first = true; + QString buffer; + QTextStream ts(&buffer); + ScImageEffectList::const_iterator i = effectsInUse.begin(); + while (i != effectsInUse.end()) + { + if (first) + first = false; + else + ts << "/"; + ts << i->effectCode << ":" << i->effectParameters; + i++; + } + return buffer; +} + bool PageItem::loadImage(const QString& filename, const bool reload, const int gsResolution, bool showMsg) { if (! asImageFrame()) @@ -5296,7 +5315,12 @@ CMSettings cms(m_Doc, IProfile, IRender); cms.setUseEmbeddedProfile(UseEmbedded); cms.allowSoftProofing(true); - if (!pixm.loadPicture(filename, pixm.imgInfo.actualPageNumber, cms, ScImage::RGBData, gsRes, &dummy, showMsg)) + ScImageCacheProxy imgcache(filename); + imgcache.addModifier("lowResType", QString::number(pixm.imgInfo.lowResType)); + if (!effectsInUse.isEmpty()) + imgcache.addModifier("effectsInUse", getImageEffectsModifier()); + bool fromCache = false; + if (!pixm.loadPicture(imgcache, fromCache, pixm.imgInfo.actualPageNumber, cms, ScImage::RGBData, gsRes, &dummy, showMsg)) { Pfile = fi.absoluteFilePath(); PictureIsAvailable = false; @@ -5354,11 +5378,24 @@ } BBoxX = pixm.imgInfo.BBoxX; BBoxH = pixm.imgInfo.BBoxH; - OrigW = pixm.width(); - OrigH = pixm.height(); + if (fromCache) + { + OrigW = imgcache.getInfo("OrigW").toInt(); + OrigH = imgcache.getInfo("OrigH").toInt(); + } + else + { + OrigW = pixm.width(); + OrigH = pixm.height(); + imgcache.addInfo("OrigW", QString::number(OrigW)); + imgcache.addInfo("OrigH", QString::number(OrigH)); + } isRaster = !(extensionIndicatesPDF(ext) || extensionIndicatesEPSorPS(ext)); if (!isRaster) + { effectsInUse.clear(); + imgcache.delModifier("effectsInUse"); + } UseEmbedded=pixm.imgInfo.isEmbedded; if (pixm.imgInfo.isEmbedded) { @@ -5368,7 +5405,7 @@ else IProfile = pixm.imgInfo.profileName; } - if (PictureIsAvailable) + if (PictureIsAvailable && !fromCache) { if ((pixm.imgInfo.colorspace == ColorSpaceDuotone) && (pixm.imgInfo.duotoneColors.count() != 0) && (!reload)) { @@ -5490,6 +5527,7 @@ ef.effectParameters = efVal; } effectsInUse.append(ef); + imgcache.addModifier("effectsInUse", getImageEffectsModifier()); } pixm.applyEffect(effectsInUse, m_Doc->PageColors, false); // if (reload) @@ -5506,9 +5544,17 @@ double ratio = pixels / 3000000.0; scaling *= sqrt(ratio); } - pixm.createLowRes(scaling); - pixm.imgInfo.lowResScale = scaling; + if (pixm.createLowRes(scaling)) + { + pixm.imgInfo.lowResScale = scaling; + pixm.saveCache(imgcache); + } + else + pixm.imgInfo.lowResScale = 1.0; } + } + if (PictureIsAvailable) + { if ((m_Doc->view()->m_canvas->usePreviewVisual())) { VisionDefectColor defect; Index: scribus/scimagecachemanager.h =================================================================== --- scribus/scimagecachemanager.h (revision 0) +++ scribus/scimagecachemanager.h (revision 0) @@ -0,0 +1,200 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#ifndef SCIMAGECACHEMANAGER_H +#define SCIMAGECACHEMANAGER_H + +#include <QList> +#include <QString> +#include <QDebug> + +#include "scribusapi.h" +#include "scimagecachedir.h" + +class QTemporaryFile; +class QFileInfo; +class ScImageCacheFile; + +/** + * @brief Scribus image cache manager + * @author Marcus Holland-Moritz + */ +class SCRIBUS_API ScImageCacheManager : public QObject +{ + Q_OBJECT + +public: + typedef ScImageCacheDir::AccessCounter AccessCounter; + + /** + * @brief Get image cache manager instance + * @return Reference to the singleton instance + */ + static ScImageCacheManager & instance(); + /** + * @brief Convert relative to absolute path + * @param fn Path relative to the image cache root directory + * @return Absolute path + */ + static QString absolutePath(const QString & fn); + + /** + * @brief Enable/disable the image cache + * @param enableCache \c true if the cache should be enabled + */ + void setEnabled(bool enableCache); + /** + * @brief Check if the image cache is enabled + * @return \c true if the cache is be enabled, \c false otherwise + */ + bool enabled(void) const { return m_isEnabled; } + /** + * @brief Set cache size limit + * @param maxCacheSizeMiB Maximum cache size in MiB. + * @return \c true if the cache size limit could be set, \c false otherwise + */ + bool setMaxCacheSizeMiB(int maxCacheSizeMiB); + /** + * @brief Set cache entry limit + * @param maxCacheEntries Maximum number of meta files in the cache + * @return \c true if the cache entry limit could be set, \c false otherwise + */ + bool setMaxCacheEntries(int maxCacheEntries); + /** + * @brief Set cache image file compression level + * @param level Image compression level. -1 is the default compression level + * for PNG images. 0 is no compression, 1 is fastest comression and + * 9 is best compression. + * @return \c true if the compression level could be set, \c false otherwise + */ + bool setCompressionLevel(int level); + /** + * @brief Get cache image file compression level + * @return Current compression level + */ + int compressionLevel() const; + + /** + * @brief Initialize the cache manager + */ + void initialize(); + /** + * @brief Try to run a cache cleanup + */ + void tryCleanup(); + /** + * @brief Try to acquire a write lock + * @return \c true if the write lock could be acquired, \c false otherwise + */ + bool acquireWriteLock(); + /** + * @brief Release a write lock + * @return \c true if the write lock could be released, \c false otherwise + */ + bool releaseWriteLock(); + /** + * @brief Remove master lock + * + * This method should only be called if a Scribus crash is detected. It + * will force the release of an existing master lock in order not to block + * other Scribus instances. + */ + void removeMasterLock(); + + /** + * @brief Access update notification + * + * This method notifies the cache manager of an updated \c access file. + * + * @param dir Path of updated directory relative to the image cache root directory + * @param from Previous access count + * @param to new access count + * @return \c true if the access count could be updated, \c false otherwise + */ + bool updateAccess(const QString & dir, AccessCounter from, AccessCounter to); + /** + * @brief File update notification + * + * This method notifies the cache manager of an updated (i.e. newly created, + * changed or removed) file in the image cache. + * + * @param file Path of updated file relative to the image cache root directory + * @return \c true if the file information could be updated, \c false otherwise + */ + bool updateFile(const QString & file); + +private slots: + void fileCreated(ScImageCacheFile * file, const QFileInfo & info); + void fileChanged(ScImageCacheFile * file, const QFileInfo & info); + void fileRemoved(ScImageCacheFile * file); + +private: + class MetaAgeList + { + public: + MetaAgeList(); + void insert(ScImageCacheFile *p); + void update(ScImageCacheFile *p, const QFileInfo & newInfo); + void remove(ScImageCacheFile *p); + ScImageCacheFile *getOldest(); + int count() const { return m_fa.size(); } + + private: + typedef QList<ScImageCacheFile *> FAL; + FAL m_fa; + + static bool ageLessThan(const ScImageCacheFile *a, const ScImageCacheFile *b); + }; + + ScImageCacheManager(); + ~ScImageCacheManager(); + + static void create(); + static void cleanupLockDir(); + static bool createLockDir(); + static QString lockDir(); + static QString masterLockFile(); + static QString writeLockTemplate(); + + void sanitizeCache(); + void updateCache(); + void cleanupCache(); + ScImageCacheFile *getOldestCacheEntry(); + + bool acquireMasterLock(); + bool releaseMasterLock(); + + bool m_isEnabled; + bool m_haveMasterLock; + bool m_inCleanup; + int m_writeLockCount; + int m_compressionLevel; + int m_maxEntries; + int m_maxSizeMiB; + qint64 m_maxTotalSize; + qint64 m_totalCacheSize; + + MetaAgeList m_metaAge; + + QTemporaryFile *m_writeLockFile; + ScImageCacheDir *m_root; +}; + +#endif Index: scribus/scimage.cpp =================================================================== --- scribus/scimage.cpp (revision 14532) +++ scribus/scimage.cpp (working copy) @@ -37,6 +37,7 @@ #include "commonstrings.h" #include "exif.h" #include "sccolorengine.h" +#include "scimagecacheproxy.h" #include "scstreamfilter.h" #include "util.h" #include "util_color.h" @@ -1185,14 +1186,17 @@ } } -void ScImage::createLowRes(double scale) +bool ScImage::createLowRes(double scale) { int w = qRound(width() / scale); int h = qRound(height() / scale); + if (w >= width() && h >= height()) // don't do unnecessary scaling + return false; QImage tmp = scaled(w, h, Qt::IgnoreAspectRatio, Qt::SmoothTransformation); if (tmp.format() != QImage::Format_ARGB32) tmp = tmp.convertToFormat(QImage::Format_ARGB32); QImage::operator=(tmp); + return true; } bool ScImage::convert2JPG(QString fn, int Quality, bool isCMYK, bool isGray) @@ -1959,6 +1963,60 @@ } } +void ScImage::addProfileToCacheModifiers(ScImageCacheProxy & cache, const QString & prefix, const ScColorProfile & profile) const +{ + if (profile) + { + cache.addModifier(prefix + "ProfileDescription", profile.productDescription()); + const ScColorProfileData *pd = profile.data(); + if (pd) + { + QString hash = pd->dataHash(); + if (!hash.isEmpty()) + cache.addModifier(prefix + "ProfileHash", hash); + } + } +} + +bool ScImage::loadPicture(ScImageCacheProxy & cache, bool & fromCache, int page, const CMSettings& cmSettings, + RequestType requestType, int gsRes, bool *realCMYK, bool showMsg) +{ + if (cache.enabled()) + { + ScColorMgmtEngine engine(cmSettings.doc() ? cmSettings.doc()->colorEngine : ScCore->defaultEngine); + cache.addModifier("cmEngineID", QString::number(engine.engineID())); + cache.addModifier("cmEngineDescription", engine.description()); + cache.addModifier("useEmbeddedProfile", QString::number(static_cast<int>(cmSettings.useEmbeddedProfile()))); + cache.addModifier("softProofingAllowed", QString::number(static_cast<int>(cmSettings.softProofingAllowed()))); + cache.addModifier("requestType", QString::number(static_cast<int>(requestType))); + cache.addModifier("gsRes", QString::number(gsRes)); + cache.addModifier("useColorManagement", QString::number(static_cast<int>(cmSettings.useColorManagement()))); + cache.addModifier("doSoftProofing", QString::number(static_cast<int>(cmSettings.doSoftProofing()))); + cache.addModifier("doGamutCheck", QString::number(static_cast<int>(cmSettings.doGamutCheck()))); + cache.addModifier("useBlackPoint", QString::number(static_cast<int>(cmSettings.useBlackPoint()))); + cache.addModifier("imageRenderingIntent", QString::number(static_cast<int>(cmSettings.imageRenderingIntent()))); + addProfileToCacheModifiers(cache, "monitor", cmSettings.monitorProfile()); + addProfileToCacheModifiers(cache, "printer", cmSettings.printerProfile()); + + fromCache = imgInfo.lowResType != 0 && cache.canUseCachedImage() && cache.load(*this) && imgInfo.deserialize(cache); + + if (fromCache) + { + cache.touch(); + return true; + } + } + else + fromCache = false; + + return loadPicture(cache.getFilename(), page, cmSettings, requestType, gsRes, realCMYK, showMsg); +} + +bool ScImage::saveCache(ScImageCacheProxy & cache) +{ + return cache.enabled() && imgInfo.serialize(cache) && cache.save(*this); +} + bool ScImage::loadPicture(const QString & fn, int page, const CMSettings& cmSettings, RequestType requestType, int gsRes, bool *realCMYK, bool showMsg) { Index: scribus/ui/preferencesdialog.h =================================================================== --- scribus/ui/preferencesdialog.h (revision 14532) +++ scribus/ui/preferencesdialog.h (working copy) @@ -38,6 +38,7 @@ #include "ui/prefs_tableofcontents.h" #include "ui/prefs_pdfexport.h" #include "ui/prefs_documentitemattributes.h" +#include "ui/prefs_imagecache.h" class PrefsManager; class ScribusMainWindow; @@ -97,6 +98,7 @@ Prefs_TableOfContents *prefs_TableOfContents; Prefs_PDFExport *prefs_PDFExport; Prefs_DocumentItemAttributes *prefs_DocumentItemAttributes; + Prefs_ImageCache *prefs_ImageCache; QMap<QListWidgetItem*, int> stackWidgetMap; int counter; Index: scribus/ui/prefs_imagecache.cpp =================================================================== --- scribus/ui/prefs_imagecache.cpp (revision 0) +++ scribus/ui/prefs_imagecache.cpp (revision 0) @@ -0,0 +1,47 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ + +#include <QFileDialog> +#include <QString> + +#include "prefs_imagecache.h" +#include "prefsstructs.h" + +Prefs_ImageCache::Prefs_ImageCache(QWidget* parent) + : Prefs_Pane(parent) +{ + setupUi(this); + languageChange(); +} + +Prefs_ImageCache::~Prefs_ImageCache() +{ +} + +void Prefs_ImageCache::languageChange() +{ + enableImageCacheCheckBox->setToolTip( "<qt>" + tr( "Enabling the image cache will significantly speed up the loading of images. Enable the cache if you are often working on large documents with lots of images and if you have plenty of disk space in your application data directory." ) + "</qt>" ); + cacheSizeLimitSpinBox->setToolTip( "<qt>"+ tr("Limit the total size of all files in the image cache directory to this amount.")+"</qt>" ); + cacheEntryLimitSpinBox->setToolTip( "<qt>" + tr( "Limit the number of cache entries to this number." ) + "</qt>" ); + compressionLevelSpinBox->setToolTip( "<qt>" + tr( "Set the level of compression for images in the cache. Higher values result in smaller cache files but also make writes to the cache slower." ) + "</qt>" ); +} + +void Prefs_ImageCache::restoreDefaults(struct ApplicationPrefs *prefsData) +{ + enableImageCacheCheckBox->setChecked(prefsData->imageCachePrefs.cacheEnabled); + cacheSizeLimitSpinBox->setValue(prefsData->imageCachePrefs.maxCacheSizeMiB); + cacheEntryLimitSpinBox->setValue(prefsData->imageCachePrefs.maxCacheEntries); + compressionLevelSpinBox->setValue(prefsData->imageCachePrefs.compressionLevel); +} + +void Prefs_ImageCache::saveGuiToPrefs(struct ApplicationPrefs *prefsData) const +{ + prefsData->imageCachePrefs.cacheEnabled = enableImageCacheCheckBox->isChecked(); + prefsData->imageCachePrefs.maxCacheSizeMiB = cacheSizeLimitSpinBox->value(); + prefsData->imageCachePrefs.maxCacheEntries = cacheEntryLimitSpinBox->value(); + prefsData->imageCachePrefs.compressionLevel = compressionLevelSpinBox->value(); +} Index: scribus/ui/prefs_imagecachebase.ui =================================================================== --- scribus/ui/prefs_imagecachebase.ui (revision 0) +++ scribus/ui/prefs_imagecachebase.ui (revision 0) @@ -0,0 +1,249 @@ +<?xml version="1.0" encoding="UTF-8"?> +<ui version="4.0"> + <class>Prefs_ImageCache</class> + <widget class="QWidget" name="Prefs_ImageCache"> + <property name="geometry"> + <rect> + <x>0</x> + <y>0</y> + <width>582</width> + <height>277</height> + </rect> + </property> + <property name="windowTitle"> + <string>Form</string> + </property> + <layout class="QVBoxLayout" name="verticalLayout"> + <item> + <widget class="QLabel" name="label"> + <property name="font"> + <font> + <pointsize>14</pointsize> + <weight>75</weight> + <bold>true</bold> + </font> + </property> + <property name="text"> + <string>Image Cache</string> + </property> + </widget> + </item> + <item> + <widget class="Line" name="line"> + <property name="orientation"> + <enum>Qt::Horizontal</enum> + </property> + </widget> + </item> + <item> + <widget class="QScrollArea" name="scrollArea"> + <property name="widgetResizable"> + <bool>true</bool> + </property> + <widget class="QWidget" name="scrollAreaWidgetContents"> + <property name="geometry"> + <rect> + <x>0</x> + <y>0</y> + <width>560</width> + <height>218</height> + </rect> + </property> + <layout class="QVBoxLayout" name="verticalLayout_2"> + <item> + <widget class="QCheckBox" name="enableImageCacheCheckBox"> + <property name="text"> + <string>Enable Image Cache</string> + </property> + </widget> + </item> + <item> + <layout class="QGridLayout" name="gridLayout"> + <item row="4" column="0"> + <widget class="QLabel" name="cacheEntryLimitLabel"> + <property name="text"> + <string>Cache Entry Limit</string> + </property> + <property name="alignment"> + <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set> + </property> + <property name="wordWrap"> + <bool>false</bool> + </property> + </widget> + </item> + <item row="4" column="1"> + <widget class="QSpinBox" name="cacheEntryLimitSpinBox"> + <property name="sizePolicy"> + <sizepolicy hsizetype="Preferred" vsizetype="Fixed"> + <horstretch>0</horstretch> + <verstretch>0</verstretch> + </sizepolicy> + </property> + <property name="minimumSize"> + <size> + <width>100</width> + <height>0</height> + </size> + </property> + <property name="alignment"> + <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set> + </property> + <property name="minimum"> + <number>100</number> + </property> + <property name="maximum"> + <number>100000</number> + </property> + <property name="singleStep"> + <number>100</number> + </property> + <property name="value"> + <number>1000</number> + </property> + </widget> + </item> + <item row="2" column="0"> + <widget class="QLabel" name="cacheSizeLimitLabel"> + <property name="text"> + <string>Cache Size Limit</string> + </property> + <property name="alignment"> + <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set> + </property> + <property name="wordWrap"> + <bool>false</bool> + </property> + </widget> + </item> + <item row="2" column="1"> + <widget class="QSpinBox" name="cacheSizeLimitSpinBox"> + <property name="sizePolicy"> + <sizepolicy hsizetype="Preferred" vsizetype="Fixed"> + <horstretch>0</horstretch> + <verstretch>0</verstretch> + </sizepolicy> + </property> + <property name="minimumSize"> + <size> + <width>100</width> + <height>0</height> + </size> + </property> + <property name="alignment"> + <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set> + </property> + <property name="buttonSymbols"> + <enum>QAbstractSpinBox::UpDownArrows</enum> + </property> + <property name="suffix"> + <string> MiB</string> + </property> + <property name="minimum"> + <number>100</number> + </property> + <property name="maximum"> + <number>1000000</number> + </property> + <property name="singleStep"> + <number>100</number> + </property> + <property name="value"> + <number>1000</number> + </property> + </widget> + </item> + <item row="2" column="2"> + <spacer name="horizontalSpacer"> + <property name="orientation"> + <enum>Qt::Horizontal</enum> + </property> + <property name="sizeHint" stdset="0"> + <size> + <width>40</width> + <height>20</height> + </size> + </property> + </spacer> + </item> + <item row="4" column="2"> + <spacer name="horizontalSpacer_2"> + <property name="orientation"> + <enum>Qt::Horizontal</enum> + </property> + <property name="sizeHint" stdset="0"> + <size> + <width>40</width> + <height>20</height> + </size> + </property> + </spacer> + </item> + <item row="5" column="0"> + <widget class="QLabel" name="compressionLevelLabel"> + <property name="text"> + <string>Compression Level</string> + </property> + <property name="alignment"> + <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set> + </property> + </widget> + </item> + <item row="5" column="1"> + <widget class="QSpinBox" name="compressionLevelSpinBox"> + <property name="sizePolicy"> + <sizepolicy hsizetype="Preferred" vsizetype="Fixed"> + <horstretch>0</horstretch> + <verstretch>0</verstretch> + </sizepolicy> + </property> + <property name="minimumSize"> + <size> + <width>100</width> + <height>0</height> + </size> + </property> + <property name="alignment"> + <set>Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter</set> + </property> + <property name="minimum"> + <number>0</number> + </property> + <property name="maximum"> + <number>9</number> + </property> + <property name="value"> + <number>6</number> + </property> + </widget> + </item> + </layout> + </item> + <item> + <spacer name="verticalSpacer"> + <property name="orientation"> + <enum>Qt::Vertical</enum> + </property> + <property name="sizeHint" stdset="0"> + <size> + <width>20</width> + <height>27</height> + </size> + </property> + </spacer> + </item> + </layout> + </widget> + </widget> + </item> + </layout> + </widget> + <tabstops> + <tabstop>scrollArea</tabstop> + <tabstop>enableImageCacheCheckBox</tabstop> + <tabstop>cacheSizeLimitSpinBox</tabstop> + <tabstop>cacheEntryLimitSpinBox</tabstop> + </tabstops> + <resources/> + <connections/> +</ui> Index: scribus/ui/prefs_imagecache.h =================================================================== --- scribus/ui/prefs_imagecache.h (revision 0) +++ scribus/ui/prefs_imagecache.h (revision 0) @@ -0,0 +1,29 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ + +#ifndef PREFS_IMAGECACHE_H +#define PREFS_IMAGECACHE_H + +#include "ui_prefs_imagecachebase.h" +#include "prefs_pane.h" +#include "scribusapi.h" + +class SCRIBUS_API Prefs_ImageCache : public Prefs_Pane, Ui::Prefs_ImageCache +{ + Q_OBJECT + + public: + Prefs_ImageCache(QWidget* parent=0); + ~Prefs_ImageCache(); + virtual void restoreDefaults(struct ApplicationPrefs *prefsData); + virtual void saveGuiToPrefs(struct ApplicationPrefs *prefsData) const; + + public slots: + void languageChange(); +}; + +#endif // PREFS_PATHS_H Index: scribus/ui/preferencesdialog.cpp =================================================================== --- scribus/ui/preferencesdialog.cpp (revision 14532) +++ scribus/ui/preferencesdialog.cpp (working copy) @@ -73,6 +73,8 @@ addItem( tr("Short Words"), loadIcon("tools.png"), prefs_ShortWords); prefs_Scripter = new Prefs_Scripter(this); addItem( tr("Scripter"), loadIcon("tools.png"), prefs_Scripter); + prefs_ImageCache = new Prefs_ImageCache(this); + addItem( tr("Image Cache"), loadIcon("tools.png"), prefs_ImageCache); arrangeIcons(); preferencesTypeList->item(0)->setSelected(true); @@ -125,6 +127,7 @@ prefs_ColorManagement->setProfiles(&localPrefs, &ScCore->InputProfiles, &ScCore->InputProfilesCMYK, &ScCore->PrinterProfiles, &ScCore->MonitorProfiles); prefs_Scrapbook->restoreDefaults(&localPrefs); prefs_Display->restoreDefaults(&localPrefs); + prefs_ImageCache->restoreDefaults(&localPrefs); } @@ -144,6 +147,7 @@ prefs_ColorManagement->saveGuiToPrefs(&localPrefs); prefs_Scrapbook->saveGuiToPrefs(&localPrefs); prefs_Display->saveGuiToPrefs(&localPrefs); + prefs_ImageCache->saveGuiToPrefs(&localPrefs); } void PreferencesDialog::applyButtonClicked() Index: scribus/scimagecacheproxy.cpp =================================================================== --- scribus/scimagecacheproxy.cpp (revision 0) +++ scribus/scimagecacheproxy.cpp (revision 0) @@ -0,0 +1,789 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#include <QCryptographicHash> +#include <QXmlStreamWriter> +#include <QXmlStreamReader> +#include <QByteArray> +#include <QDateTime> +#include <QDir> +#include <QFile> +#include <QFileInfo> + +#include "sclockedfile.h" +#include "scimagecacheproxy.h" +#include "scimagecachemanager.h" +#include "scimagecachewriteaction.h" +#include "scpaths.h" +#include "util_file.h" + +#define SC_DEBUG_FILE defined(DEBUG_SCIMAGECACHE) +#include "scdebug.h" + +// MD5 has been chosen as a hash algorithm as it less prone to collisions than MD4, +// but at the same time twice as fast to compute as SHA-1. Furthermore, it's 32 bits +// shorter than SHA-1, making the filenames at least a little shorter. + +namespace { + const QString CACHEFILE_VERSION("1"); + const QCryptographicHash::Algorithm HASH_ALGORITHM = QCryptographicHash::Md5; + const int CACHEDIR_LEVELS = 2; + const char * const imageFormat = "PNG"; + + inline QString absolutePath(const QString & fn) + { + return ScImageCacheManager::absolutePath(fn); + } +} + +const QString ScImageCacheProxy::metaSuffix("xml"); +const QString ScImageCacheProxy::referenceSuffix("ref"); +const QString ScImageCacheProxy::imageSuffix("png"); + +ScImageCacheProxy::ScImageCacheProxy(const QString & fn) + : m_filename(fn), m_isEnabled(ScImageCacheManager::instance().enabled()) +{ + if (!m_isEnabled) + return; + + QFileInfo imfo(m_filename); + + if (imfo.exists()) + { + addMetadata("version", CACHEFILE_VERSION); + addMetadata("path", m_filename); + addMetadata("size", QString::number(imfo.size())); + addMetadata("lastModifiedUTC", imfo.lastModified().toUTC().toString(Qt::ISODate)); + } +} + +ScImageCacheProxy::~ScImageCacheProxy() +{ + // nothing :) +} + +void ScImageCacheProxy::addMetadata(const QString & key, const QString & value) +{ + m_metadata[key] = value; +} + +void ScImageCacheProxy::addModifier(const QString & key, const QString & value) +{ + m_modifier[key] = value; + m_metanameCache.clear(); +} + +void ScImageCacheProxy::delModifier(const QString & key) +{ + m_modifier.remove(key); + m_metanameCache.clear(); +} + +void ScImageCacheProxy::addInfo(const QString & key, const QString & value) +{ + m_imginfo[key] = value; +} + +QString ScImageCacheProxy::getInfo(const QString & key) const +{ + return m_imginfo[key]; +} + +QString ScImageCacheProxy::imageFile(const QString & base) +{ + return base + "." + imageSuffix; +} + +QString ScImageCacheProxy::referenceFile(const QString & base) +{ + return base + "." + referenceSuffix; +} + +QString ScImageCacheProxy::getBaseName(const QString & metafile) +{ + QString base; + return loadMetadata(metafile, 0, 0, 0, &base) ? base : QString(); +} + +bool ScImageCacheProxy::loadMetadata(ScLockedFile *file, MetaMap *meta, MetaMap *mod, MetaMap *info, QString *base) +{ + QXmlStreamReader xml(file->io()); + + bool baseFound = false; + bool metaFound = false; + bool modFound = false; + bool infoFound = false; + + while (!xml.atEnd()) + { + if (xml.readNext() == QXmlStreamReader::StartElement) + { + QXmlStreamAttributes attr = xml.attributes(); + + if (xml.name() == "cache") + { + if (attr.hasAttribute("base")) + { + if (base) + *base = attr.value("base").toString(); + + baseFound = true; + } + } + else if (xml.name() == "metadata") + { + if (meta) + { + meta->clear(); + + foreach (QXmlStreamAttribute a, attr) + (*meta)[a.name().toString()] = a.value().toString(); + } + + metaFound = true; + } + else if (xml.name() == "modifier") + { + if (mod) + { + mod->clear(); + + foreach (QXmlStreamAttribute a, attr) + (*mod)[a.name().toString()] = a.value().toString(); + } + + modFound = true; + } + else if (xml.name() == "imginfo") + { + if (info) + { + info->clear(); + + foreach (QXmlStreamAttribute a, attr) + (*info)[a.name().toString()] = a.value().toString(); + } + + infoFound = true; + } + } + } + + if (xml.hasError()) + { + scDebug() << "error parsing" << file->name() << xml.errorString() << "in line" << xml.lineNumber() << "column" << xml.columnNumber(); + return false; + } + + if (!baseFound) scDebug() << "base not found"; + if (!metaFound) scDebug() << "meta not found"; + if (!modFound) scDebug() << "mod not found"; + if (!infoFound) scDebug() << "info not found"; + + return baseFound && metaFound && modFound && infoFound; +} + +bool ScImageCacheProxy::loadMetadata(const QString & fn, MetaMap *meta, MetaMap *mod, MetaMap *info, QString *base) +{ + ScLockedFileRO file(absolutePath(fn)); + if (!file.open()) + { + scDebug() << "failed to open" << fn; + return false; + } + return loadMetadata(&file, meta, mod, info, base); +} + +bool ScImageCacheProxy::loadMetadata(MetaMap *meta, MetaMap *mod, MetaMap *info, QString *base) const +{ + return loadMetadata(metaName(), meta, mod, info, base); +} + +void ScImageCacheProxy::saveMetadata(ScLockedFile *file, const MetaMap & meta, const MetaMap & mod, const MetaMap & info, const QString & base) +{ + QXmlStreamWriter xml(file->io()); + + xml.setAutoFormatting(true); + xml.writeStartDocument(); + xml.writeStartElement("cache"); + xml.writeAttribute("base", base); + xml.writeStartElement("metadata"); + for (MetaMap::const_iterator i = meta.constBegin(); i != meta.constEnd(); i++) + xml.writeAttribute(i.key(), i.value()); + xml.writeEndElement(); + xml.writeStartElement("modifier"); + for (MetaMap::const_iterator i = mod.constBegin(); i != mod.constEnd(); i++) + xml.writeAttribute(i.key(), i.value()); + xml.writeEndElement(); + xml.writeStartElement("imginfo"); + for (MetaMap::const_iterator i = info.constBegin(); i != info.constEnd(); i++) + xml.writeAttribute(i.key(), i.value()); + xml.writeEndElement(); + xml.writeEndElement(); + xml.writeEndDocument(); +} + +bool ScImageCacheProxy::canUseCachedImage() const +{ + if (!enabled()) + return false; + + MetaMap cmeta; // cached metadata + MetaMap cmod; // cached modifiers + QString base; + + if (m_metadata.isEmpty()) + { + scDebug() << "cannot use cached image, no metadata"; + return false; + } + + if (!loadMetadata(&cmeta, &cmod, 0, &base)) + { + scDebug() << "cannot use cached image, load metadata failed"; + return false; + } + + QString fn = absolutePath(imageFile(base)); + QFileInfo info(fn); + + if (!info.exists()) + return false; + + if (cmeta.size() != m_metadata.size()) + return false; + + if (cmod.size() != m_modifier.size()) + return false; + + for (MetaMap::const_iterator i = m_metadata.constBegin(); i != m_metadata.constEnd(); i++) + if (cmeta[i.key()] != i.value()) + return false; + + for (MetaMap::const_iterator i = m_modifier.constBegin(); i != m_modifier.constEnd(); i++) + if (cmod[i.key()] != i.value()) + return false; + + return true; +} + +QString ScImageCacheProxy::addDirLevels(QString name) +{ + Q_ASSERT(name.size() > CACHEDIR_LEVELS); + if (name.size() <= CACHEDIR_LEVELS) + { + scDebug() << "BUG: invalid name" << name << "passed to addDirLevels"; + return QString(); + } + for (int i = CACHEDIR_LEVELS; i > 0; i--) + name.insert(i, '/'); + return name; +} + +QString ScImageCacheProxy::imageBaseName(const QImage & image) const +{ + if (!m_metadata.contains("size")) + { + scDebug() << "size not present in metadata"; + return QString(); + } + QCryptographicHash hash(HASH_ALGORITHM); + for (int i = 0; i < image.height(); i++) + hash.addData(reinterpret_cast<const char *>(image.scanLine(i)), image.bytesPerLine()); + return addDirLevels(hash.result().toHex()) + "-" + m_metadata["size"]; +} + +const QString & ScImageCacheProxy::metaName() const +{ + if (m_metanameCache.isEmpty()) + { + QCryptographicHash hash(HASH_ALGORITHM); + hash.addData(m_filename.toUtf8()); + for (MetaMap::const_iterator i = m_modifier.constBegin(); i != m_modifier.constEnd(); i++) + { + hash.addData(i.key().toUtf8()); + hash.addData(i.value().toUtf8()); + } + m_metanameCache = addDirLevels(hash.result().toHex()) + "." + metaSuffix; + } + return m_metanameCache; +} + +bool ScImageCacheProxy::createCacheDir() +{ + QString cachedir = ScPaths::getImageCacheDir(); + QDir cdir(cachedir); + + if (!cdir.exists()) + { + scDebug() << "creating" << cachedir; + if (!cdir.mkpath(cachedir)) + { + scDebug() << "could not create" << cachedir; + return false; + } + } + + return true; +} + +bool ScImageCacheProxy::load(QImage & image) +{ + if (!enabled()) + return false; + + QString base; + + if (!loadMetadata(&m_metadata, &m_modifier, &m_imginfo, &base)) + { + scDebug() << "could not load metadata for" << m_filename; + return false; + } + + QString fn = absolutePath(imageFile(base)); + + if (!image.load(fn)) + { + scDebug() << "could not load cached image for" << m_filename; + return false; + } + + scDebug() << "successfully loaded" << m_filename << "from" << fn; + return true; +} + +bool ScImageCacheProxy::save(const QImage & image) +{ + if (!enabled()) + return false; + + scDebug() << "saving" << m_filename << "to cache"; + + Q_ASSERT(!m_metadata.isEmpty()); + Q_ASSERT(!m_imginfo.isEmpty()); + + if (m_metadata.isEmpty()) + { + scDebug() << "BUG: attempt to save cache without metadata"; + return false; + } + + if (m_imginfo.isEmpty()) + { + scDebug() << "BUG: attempt to save cache without image info"; + return false; + } + + if (!createCacheDir()) + return false; + + // The cache write lock does not prevent other instances from writing to + // the cache. It only prevents other instances from setting a master lock. + + ScImageCacheWriteAction action; + + if (!action.start()) + return false; + + // Computing the imageBaseName is rather longish, so do it before locking + // the files in order to keep the lock time as short as possible. + + QString base = imageBaseName(image); + + Q_ASSERT(!base.isEmpty()); + + if (base.isEmpty()) + { + scDebug() << "BUG: could not create image base name"; + return false; + } + + scDebug() << "storing as base" << base; + + QString refName = base + "." + referenceSuffix; + QString imgName = base + "." + imageSuffix; + QString oldBase; + QString oldRefName; + QString oldImgName; + bool haveOldMeta = false; + bool haveOldRef = false; + + ScLockedFileRW meta(absolutePath(metaName())); + ScLockedFileRW ref(absolutePath(refName)); + ScLockedFileRW img(absolutePath(imgName)); + ScLockedFileRW oldRef; + + if (!meta.createPath()) + { + scDebug() << "could not create path for" << meta.name(); + return false; + } + + if (!ref.createPath()) + { + scDebug() << "could not create path for" << ref.name(); + return false; + } + + // Try to acquire necessary locks. Locks will be automatically cleaned up + // upon destruction of the lock object, so we can safely return at any time. + + if (!action.add(metaName())) + { + scDebug() << "could not add lock for" << metaName(); + return false; + } + + // This is a bit tricky... if the meta file already exists, it will most + // probably point to a different reference file. We need to access this + // "old" reference file as well in order to decrement its reference count. + + if (meta.exists()) + { + if (!loadMetadata(0, 0, 0, &oldBase)) + { + scDebug() << "could not read metadata from" << meta.name(); + return false; + } + + haveOldMeta = true; + oldRefName = oldBase + "." + referenceSuffix; + oldImgName = oldBase + "." + imageSuffix; + + if (oldBase != base) + { + oldRef.setName(absolutePath(oldRefName)); + + if (!action.add(oldRefName)) + { + scDebug() << "could not add" << oldRefName << "to action"; + return false; + } + if (!action.add(oldImgName)) + { + scDebug() << "could not add" << oldImgName << "to action"; + return false; + } + + haveOldRef = oldRef.exists(); + + if (!haveOldRef) + oldRef.unlock(); + } + } + + if (!action.add(refName)) + { + scDebug() << "could not add" << refName << "to action"; + return false; + } + + if (!action.add(imgName)) + { + scDebug() << "could not add" << imgName << "to action"; + return false; + } + + // The meta and reference files have both been locked now, so we're safe to + // write to the cache. Locking the reference file implicitly also locks the + // image file. We can also safely open all files already, as they are only + // temporary files and don't conflict with other files in the cache. + + // cases: + // * completely new entry, none of the files exist + // - create image file + // - create reference file with refcount 1 + // - update meta file + // * new meta file, but reference file exists + // - increment reference count + // - update meta file + // - keep image + // * old meta file exists and reference files are identical + // - keep reference file + // - update meta file + // - keep image + // * old meta file exists and reference files differ + // - decrement reference count of old reference file + // - continue as above + + // Open the metafile. If this fails, everything else is quite useless. + + if (!meta.open()) + { + scDebug() << "could not open meta file" << meta.name(); + return false; + } + + // Update the reference files if necessary. + + if (haveOldRef) + { + // we don't care if this fails + // if there's any problem, the next cache cleanup will detect it + unrefImage(&oldRef, oldImgName); + } + + if (oldBase != base) + { + if (!refImage(&ref)) + { + scDebug() << "could not reference new image" << ref.name(); + return false; + } + } + + // Write image file if necessary. Existing image files are *never* re-written + // under the assumption that there are no collisions. + + if (img.exists()) + { + scDebug() << "cached image for" << m_filename << "already exists in" << img.name(); + } + else + { + if (!img.open()) + { + scDebug() << "could not open image file" << img.name(); + return false; + } + int level = ScImageCacheManager::instance().compressionLevel(); + level = level < 0 ? level : 10*(9 - level); + scDebug() << "compressing" << imageFormat << "image, quality =" << level; + if (!image.save(img.io(), imageFormat, level)) + { + scDebug() << "could not save image" << img.name(); + return false; + } + + img.commit(); + + scDebug() << "successfully stored" << m_filename << "in cache as" << img.name(); + } + + // Save the metadata. + + saveMetadata(&meta, m_metadata, m_modifier, m_imginfo, base); + meta.commit(); + + // Explicit commit will also trigger access file update + + action.commit(); + + return true; +} + +bool ScImageCacheProxy::loadRef(ScLockedFile *file, int & refcount) +{ + QXmlStreamReader xml(file->io()); + bool refcountFound = false; + + while (!xml.atEnd()) + { + if (xml.readNext() == QXmlStreamReader::StartElement) + { + QXmlStreamAttributes attr = xml.attributes(); + + if (xml.name() == "reference") + if (attr.hasAttribute("count")) + refcount = attr.value("count").toString().toInt(&refcountFound); + } + } + + if (xml.hasError()) + { + scDebug() << "error parsing" << file->name() << xml.errorString() << "in line" << xml.lineNumber() << "column" << xml.columnNumber(); + return false; + } + + return refcountFound; +} + +void ScImageCacheProxy::saveRef(ScLockedFile *file, int refcount) +{ + QXmlStreamWriter xml(file->io()); + + xml.setAutoFormatting(true); + xml.writeStartDocument(); + xml.writeStartElement("reference"); + xml.writeAttribute("count", QString::number(refcount)); + xml.writeEndElement(); + xml.writeEndDocument(); +} + +bool ScImageCacheProxy::getRefCount(const QString & reffile, int & refcount) +{ + return getRefCountAbs(absolutePath(reffile), refcount); +} + +bool ScImageCacheProxy::getRefCountAbs(const QString & reffile, int & refcount) +{ + ScLockedFileRO ro(reffile); + if (!ro.open()) + { + scDebug() << "could not open reference file" << ro.name(); + return false; + } + if (!loadRef(&ro, refcount)) + { + scDebug() << "could not read reference file" << ro.name(); + return false; + } + return true; +} + +bool ScImageCacheProxy::fixRefCount(const QString & reffile, int refcount) +{ + ScLockedFileRW rw(absolutePath(reffile)); + if (!rw.open()) + { + scDebug() << "could not open reference file" << rw.name(); + return false; + } + saveRef(&rw, refcount); + return rw.commit(); +} + +bool ScImageCacheProxy::removeCacheEntry(const QString & metafile, bool haveMasterLock) +{ + ScImageCacheWriteAction action(haveMasterLock); + + if (!action.start()) + return false; + + ScLockedFileRW meta(absolutePath(metafile)); + + if (!action.add(metafile)) + { + scDebug() << "could not add" << metafile; + return false; + } + + QString base = getBaseName(metafile); + + meta.remove(); + + if (base.isEmpty()) + { + scDebug() << "empty basename in" << metafile; + } + else + { + QString reffile = referenceFile(base); + QString imgfile = imageFile(base); + + if (!action.add(reffile)) + { + scDebug() << "could not add" << reffile; + return false; + } + + if (!action.add(imgfile)) + { + scDebug() << "could not add" << imgfile; + return false; + } + + ScLockedFileRW ref(absolutePath(reffile)); + + // we don't care if these fail + // if there's any problem, the next cache cleanup will detect it + unrefImage(&ref, absolutePath(imgfile)); + } + + action.commit(); + + return true; +} + +bool ScImageCacheProxy::refImage(ScLockedFile *file) +{ + int refcount = 0; + + if (file->exists() && !getRefCountAbs(file->name(), refcount)) + return false; + + refcount++; + + if (!file->open()) + { + scDebug() << "could not open reference file for writing" << file->name(); + return false; + } + + saveRef(file, refcount); + + return file->commit(); +} + +bool ScImageCacheProxy::unrefImage(ScLockedFile *file, const QString & imageName) +{ + int refcount = 0; + + if (file->exists()) + { + if (!getRefCountAbs(file->name(), refcount)) + return false; + } + else + { + // could also happen if someone else is messing with the cache + scDebug() << "BUG: attempt to unref non-existent reference file" << file->name(); + return false; + } + + refcount--; + + if (refcount == 0) + { + bool rv = true; + + scDebug() << "refcount dropped to zero for" << file->name(); + + if (!file->remove()) + { + scDebug() << "could not remove reference file" << file->name(); + rv = false; + } + + if (QFile::exists(imageName) && !QFile::remove(imageName)) + { + scDebug() << "could not remove image file" << imageName; + rv = false; + } + + return rv; + } + + if (!file->open()) + { + scDebug() << "could not open reference file for writing" << file->name(); + return false; + } + + saveRef(file, refcount); + + return file->commit(); +} + +bool ScImageCacheProxy::touch() const +{ + scDebug() << "touching metafile" << metaName(); + return touchFile(absolutePath(metaName())); +} Index: scribus/scribus.cpp =================================================================== --- scribus/scribus.cpp (revision 14532) +++ scribus/scribus.cpp (working copy) @@ -49,6 +49,12 @@ #include <QTranslator> #include <QWheelEvent> +#ifdef DEBUG_LOAD_TIMES +#include <QDebug> +#include <QTime> +#include <sys/times.h> +#endif + #include <cstdio> #include <cstdlib> #include <cassert> @@ -221,6 +227,7 @@ #include "ui/vruler.h" #include "loadsaveplugin.h" #include "plugins/formatidlist.h" +#include "scimagecachemanager.h" #if defined(_WIN32) @@ -3630,6 +3637,12 @@ bool ScribusMainWindow::loadDoc(QString fileName) { +#ifdef DEBUG_LOAD_TIMES + QTime t; + struct tms tms1, tms2; + t.start(); + times(&tms1); +#endif undoManager->setUndoEnabled(false); QFileInfo fi(fileName); if (!fi.exists()) @@ -4005,6 +4018,16 @@ qApp->changeOverrideCursor(QCursor(Qt::ArrowCursor)); undoManager->setUndoEnabled(true); doc->setModified(false); +#ifdef DEBUG_LOAD_TIMES + times(&tms2); + double ticks = sysconf(_SC_CLK_TCK); + double user = (tms2.tms_utime - tms1.tms_utime)/ticks; + double sys = (tms2.tms_stime - tms1.tms_stime)/ticks; + double cuser = (tms2.tms_cutime - tms1.tms_cutime)/ticks; + double csys = (tms2.tms_cstime - tms1.tms_cstime)/ticks; + qDebug("loaded document in %.3f seconds (%.3f user + %.3f sys = %.3f sec, child %.3f user + %.3f sys = %.3f sec)", + t.elapsed()/1000.0, user, sys, user + sys, cuser, csys, cuser + csys); +#endif return ret; } @@ -7583,6 +7606,11 @@ qWarning( "%s", message.toLocal8Bit().data() ); } } + ScImageCacheManager & icm = ScImageCacheManager::instance(); + icm.setEnabled(newPrefs.imageCachePrefs.cacheEnabled); + icm.setMaxCacheSizeMiB(newPrefs.imageCachePrefs.maxCacheSizeMiB); + icm.setMaxCacheEntries(newPrefs.imageCachePrefs.maxCacheEntries); + icm.setCompressionLevel(newPrefs.imageCachePrefs.compressionLevel); } prefsManager->SavePrefs(); Index: scribus/util_file.h =================================================================== --- scribus/util_file.h (revision 14532) +++ scribus/util_file.h (working copy) @@ -64,5 +64,14 @@ * @return true on success, false on failre. **/ bool SCRIBUS_API moveFile(const QString& source, const QString& target); +/** +* @brief Update the access and modification time of a file + * + * This function updates the access and modification time of a file. + * + * @param file name of the file to touch + * @return true on success, false on failure. +**/ +bool SCRIBUS_API touchFile(const QString& file); #endif Index: scribus/scdebug.h =================================================================== --- scribus/scdebug.h (revision 0) +++ scribus/scdebug.h (revision 0) @@ -0,0 +1,51 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +/** + * @brief A per-file debug stream based on qDebug() + * + * Define SC_DEBUG_FILE to zero (debugging disabled) or non-zero + * (debugging enabled) before including this file. Not defining it + * means enabling debug support unless QT_NO_DEBUG_OUTPUT is defined. + * Debugging will be disabled at compile time, so there's no need to + * comment all lines that generate debug output. + * + * @author Marcus Holland-Moritz + */ + +#if defined(QT_NO_DEBUG_OUTPUT) || (defined(SC_DEBUG_FILE) && SC_DEBUG_FILE == 0) + +class ScNoDebug +{ +public: + inline ScNoDebug() {} + inline ~ScNoDebug() {} +}; +template<typename T> +inline ScNoDebug operator<<(ScNoDebug debug, const T &) { return debug; } +inline ScNoDebug scDebug() { return ScNoDebug(); } + +#else + +#include <QDebug> +#include <QTime> +inline QDebug scDebug() { return QDebug(QtDebugMsg) << QTime::currentTime().toString("[hh:mm:ss.zzz]"); } + +#endif Index: scribus/colormgmt/sccolorprofiledata.cpp =================================================================== --- scribus/colormgmt/sccolorprofiledata.cpp (revision 0) +++ scribus/colormgmt/sccolorprofiledata.cpp (revision 0) @@ -0,0 +1,23 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ + +#include <QCryptographicHash> + +#include "sccolorprofiledata.h" + +QString ScColorProfileData::dataHash() const +{ + // This only needs to be hashed once, as the profile data doesn't change once it's set. + // <jghali> mhx: sccolorprofiledata must only be created by a sccolorengine and not changed afterwards + if (m_profileDataHash.isEmpty()) + { + if (m_profileData.isEmpty()) + return QString(); + m_profileDataHash = QCryptographicHash::hash(m_profileData, QCryptographicHash::Md5).toHex(); + } + return m_profileDataHash; +} Index: scribus/colormgmt/sccolorprofiledata.h =================================================================== --- scribus/colormgmt/sccolorprofiledata.h (revision 14532) +++ scribus/colormgmt/sccolorprofiledata.h (working copy) @@ -16,11 +16,15 @@ class ScColorProfileData : public ScColorMgmtElem { +private: + mutable QString m_profileDataHash; + protected: QString m_profilePath; QByteArray m_profileData; public: + QString dataHash() const; QString path() const { return m_profilePath; } Index: scribus/colormgmt/CMakeLists.txt =================================================================== --- scribus/colormgmt/CMakeLists.txt (revision 14532) +++ scribus/colormgmt/CMakeLists.txt (working copy) @@ -11,6 +11,7 @@ sccolormgmtstructs.cpp sccolorprofile.cpp sccolorprofilecache.cpp + sccolorprofiledata.cpp sccolorspacedata.cpp sccolortransform.cpp sccolortransformpool.cpp Index: scribus/sclockedfile.h =================================================================== --- scribus/sclockedfile.h (revision 0) +++ scribus/sclockedfile.h (revision 0) @@ -0,0 +1,113 @@ +/* +For general Scribus (>=1.3.2) copyright and licensing information please refer +to the COPYING file provided with the program. Following this notice may exist +a copyright and/or license notice that predates the release of Scribus 1.3.2 +for which a new license (GPL+exception) is in place. +*/ +/*************************************************************************** + copyright : (C) 2010 by Marcus Holland-Moritz + email : scribus@mhxnet.de +***************************************************************************/ + +/*************************************************************************** +* * +* This program is free software; you can redistribute it and/or modify * +* it under the terms of the GNU General Public License as published by * +* the Free Software Foundation; either version 2 of the License, or * +* (at your option) any later version. * +* * +***************************************************************************/ + +#ifndef SCLOCKEDFILE_H +#define SCLOCKEDFILE_H + +#include "scconfig.h" +#include "scribusapi.h" + +#include <QFile> +#include <QString> +#include <QTemporaryFile> + +class QFileInfo; + +/** + * @brief Base class for locked file access + * @author Marcus Holland-Moritz + */ +class ScLockedFile +{ +public: + virtual ~ScLockedFile(); + + void setName(const QString & name); + const QString & name() const { return m_fileName; } + bool createPath() const; + + bool lock(); + bool unlock(); + bool locked() const { return m_isLocked; } + + bool exists() const; + bool remove() const; + + virtual bool open() = 0; + virtual bool commit() = 0; + + virtual QIODevice *io() = 0; + + static const QString lockSuffix; + +protected: + ScLockedFile(); + ScLockedFile(const QString & name); + + QString m_fileName; + bool m_isOpened; + +private: + bool m_isLocked; + + QString lockName() const; +}; + +/** + * @brief Read-only locked file access + * @author Marcus Holland-Moritz + */ +class ScLockedFileRO : public ScLockedFile +{ +public: + ScLockedFileRO(); + ScLockedFileRO(const QString & name); + + virtual bool open(); + virtual bool commit(); + + virtual QIODevice *io() { return &m_file; } + +private: + QFile m_file; +}; + +/** + * @brief Read/write locked file access + * @author Marcus Holland-Moritz + */ +class ScLockedFileRW : public ScLockedFile +{ +public: + ScLockedFileRW(); + ScLockedFileRW(const QString & name); + + virtual bool open(); + virtual bool commit(); + + virtual QIODevice *io() { return &m_file; } + +private: + QTemporaryFile m_file; + + static QString templateName(const QFileInfo & info); +}; + +#endif Index: scribus/CMakeLists.txt =================================================================== --- scribus/CMakeLists.txt (revision 14532) +++ scribus/CMakeLists.txt (working copy) @@ -105,6 +105,7 @@ ui/prefs_fontsbase.ui ui/prefs_guidesbase.ui ui/prefs_hyphenatorbase.ui + ui/prefs_imagecachebase.ui ui/prefs_keyboardshortcutsbase.ui ui/prefs_miscellaneousbase.ui ui/prefs_pathsbase.ui @@ -295,6 +296,7 @@ ui/prefs_fonts.h ui/prefs_guides.h ui/prefs_hyphenator.h + ui/prefs_imagecache.h ui/prefs_keyboardshortcuts.h ui/prefs_miscellaneous.h ui/prefs_paths.h @@ -326,6 +328,9 @@ ui/scfilewidget.h scgtplugin.h schelptreemodel.h + scimagecachedir.h + scimagecachefile.h + scimagecachemanager.h ui/scinputdialog.h ui/scmenu.h ui/scmessagebox.h @@ -608,6 +613,7 @@ ui/prefs_fonts.cpp ui/prefs_guides.cpp ui/prefs_hyphenator.cpp + ui/prefs_imagecache.cpp ui/prefs_keyboardshortcuts.cpp ui/prefs_miscellaneous.cpp ui/prefs_paths.cpp @@ -654,6 +660,11 @@ scgzfile.cpp schelptreemodel.cpp scimage.cpp + scimagecacheproxy.cpp + scimagecachedir.cpp + scimagecachefile.cpp + scimagecachemanager.cpp + scimagecachewriteaction.cpp scimagestructs.cpp scimgdataloader.cpp scimgdataloader_gimp.cpp @@ -667,6 +678,7 @@ scimgdataloader_wpg.cpp ui/scinputdialog.cpp sclayer.cpp + sclockedfile.cpp ui/scmenu.cpp ui/scmessagebox.cpp scmimedata.cpp Index: scribus/scimagestructs.h =================================================================== --- scribus/scimagestructs.h (revision 14532) +++ scribus/scimagestructs.h (working copy) @@ -15,6 +15,8 @@ #include "fpointarray.h" #include "sccolor.h" +class ScImageCacheProxy; + struct ImageLoadRequest { bool visible; @@ -80,6 +82,10 @@ ExifValues(void); void init(void); + // Remember to increment this version number and update + // the QDataStream operators if this class in changed. + static const qint32 dsVersion; + int width; int height; float ExposureTime; @@ -121,6 +127,14 @@ ImageInfoRecord(void); void init(void); + // Remember to increment this version number and update + // the serialization routines if this class in changed. + static const int iirVersion; + + bool canSerialize() const; + bool serialize(ScImageCacheProxy & cache) const; + bool deserialize(const ScImageCacheProxy & cache); + ImageTypeEnum type; /* 0 = jpg, 1 = tiff, 2 = psd, 3 = eps/ps, 4 = pdf, 5 = jpg2000, 6 = other */ int xres; int yres; Index: scribus/scpaths.cpp =================================================================== --- scribus/scpaths.cpp (revision 14532) +++ scribus/scpaths.cpp (working copy) @@ -336,6 +336,11 @@ #endif } +QString ScPaths::getImageCacheDir(void) +{ + return getApplicationDataDir() + "cache/img/"; +} + QString ScPaths::getPluginDataDir(void) { return getApplicationDataDir() + "plugins/"; Index: scribus/util_file.cpp =================================================================== --- scribus/util_file.cpp (revision 14532) +++ scribus/util_file.cpp (working copy) @@ -7,6 +7,12 @@ #include "util_file.h" +#ifdef _MSC_VER +# include <sys/utime.h> +#else +# include <utime.h> +#endif + #include <QDataStream> #include <QFile> #include <QString> @@ -169,3 +175,13 @@ moveSucceed &= QFile::remove(source); return moveSucceed; } + +bool touchFile(const QString& file) +{ +#if defined(_WIN32) && defined(HAVE_UNICODE) + return _wutime((const wchar_t*) file.utf16(), NULL) == 0; +#else + QByteArray fname = file.toLocal8Bit(); + return utime(fname.data(), NULL) == 0; +#endif +} Index: Scribus.pro =================================================================== --- Scribus.pro (revision 14532) +++ Scribus.pro (working copy) @@ -1,12 +1,12 @@ ###################################################################### -# Automatically generated by qmake (2.01a) Wed Jan 6 20:46:08 2010 +# Automatically generated by qmake (2.01a) Sa. Jan 16 20:24:21 2010 ###################################################################### TEMPLATE = app -TARGET = +TARGET = Scribus DEPENDPATH += . \ scribus \ - scribus/colormngt \ + scribus/colormgmt \ scribus/desaxe \ scribus/designer \ scribus/fonts \ @@ -120,7 +120,7 @@ scribus \ win32/vc8 \ scribus/fonts \ - scribus/colormngt \ + scribus/colormgmt \ scribus/text \ scribus/styles \ scribus/desaxe \ @@ -301,6 +301,7 @@ scribus/sccolorengine.h \ scribus/sccolorshade.h \ scribus/scconfig.h \ + scribus/scdebug.h \ scribus/scdocoutput.h \ scribus/scdocoutput_ps2.h \ scribus/scfonts.h \ @@ -309,6 +310,11 @@ scribus/scgzfile.h \ scribus/schelptreemodel.h \ scribus/scimage.h \ + scribus/scimagecachedir.h \ + scribus/scimagecachefile.h \ + scribus/scimagecachemanager.h \ + scribus/scimagecacheproxy.h \ + scribus/scimagecachewriteaction.h \ scribus/scimagestructs.h \ scribus/scimgdataloader.h \ scribus/scimgdataloader_gimp.h \ @@ -323,6 +329,7 @@ scribus/scimgdataloader_wpg.h \ scribus/sclayer.h \ scribus/sclistboxpixmap.h \ + scribus/sclockedfile.h \ scribus/scmimedata.h \ scribus/scpageoutput.h \ scribus/scpageoutput_ps2.h \ @@ -392,21 +399,22 @@ scribus/util_text.h \ scribus/vgradient.h \ scribus/vgradientex.h \ - scribus/colormngt/sccolormngtelem.h \ - scribus/colormngt/sccolormngtengine.h \ - scribus/colormngt/sccolormngtenginedata.h \ - scribus/colormngt/sccolormngtenginefactory.h \ - scribus/colormngt/sccolormngtimplelem.h \ - scribus/colormngt/sccolormngtstructs.h \ - scribus/colormngt/sccolorprofile.h \ - scribus/colormngt/sccolorprofilecache.h \ - scribus/colormngt/sccolorprofiledata.h \ - scribus/colormngt/sccolortransform.h \ - scribus/colormngt/sccolortransformdata.h \ - scribus/colormngt/sccolortransformpool.h \ - scribus/colormngt/sclcmscolormngtengineimpl.h \ - scribus/colormngt/sclcmscolorprofileimpl.h \ - scribus/colormngt/sclcmscolortransformimpl.h \ + scribus/colormgmt/sccolormgmtelem.h \ + scribus/colormgmt/sccolormgmtengine.h \ + scribus/colormgmt/sccolormgmtenginedata.h \ + scribus/colormgmt/sccolormgmtenginefactory.h \ + scribus/colormgmt/sccolormgmtimplelem.h \ + scribus/colormgmt/sccolormgmtstructs.h \ + scribus/colormgmt/sccolorprofile.h \ + scribus/colormgmt/sccolorprofilecache.h \ + scribus/colormgmt/sccolorprofiledata.h \ + scribus/colormgmt/sccolorspacedata.h \ + scribus/colormgmt/sccolortransform.h \ + scribus/colormgmt/sccolortransformdata.h \ + scribus/colormgmt/sccolortransformpool.h \ + scribus/colormgmt/sclcmscolormgmtengineimpl.h \ + scribus/colormgmt/sclcmscolorprofileimpl.h \ + scribus/colormgmt/sclcmscolortransformimpl.h \ scribus/desaxe/actions.h \ scribus/desaxe/automata.h \ scribus/desaxe/base_actions.h \ @@ -570,6 +578,7 @@ scribus/ui/prefs_fonts.h \ scribus/ui/prefs_guides.h \ scribus/ui/prefs_hyphenator.h \ + scribus/ui/prefs_imagecache.h \ scribus/ui/prefs_itemtools.h \ scribus/ui/prefs_keyboardshortcuts.h \ scribus/ui/prefs_miscellaneous.h \ @@ -602,6 +611,7 @@ scribus/ui/scinputdialog.h \ scribus/ui/scmenu.h \ scribus/ui/scmessagebox.h \ + scribus/ui/scmwmenumanager.h \ scribus/ui/scprogressbar.h \ scribus/ui/scrapbookpalette.h \ scribus/ui/scresizecursor.h \ @@ -935,6 +945,7 @@ scribus/ui/prefs_fontsbase.ui \ scribus/ui/prefs_guidesbase.ui \ scribus/ui/prefs_hyphenatorbase.ui \ + scribus/ui/prefs_imagecachebase.ui \ scribus/ui/prefs_itemtoolsbase.ui \ scribus/ui/prefs_keyboardshortcutsbase.ui \ scribus/ui/prefs_miscellaneousbase.ui \ @@ -1112,6 +1123,11 @@ scribus/scgzfile.cpp \ scribus/schelptreemodel.cpp \ scribus/scimage.cpp \ + scribus/scimagecachedir.cpp \ + scribus/scimagecachefile.cpp \ + scribus/scimagecachemanager.cpp \ + scribus/scimagecacheproxy.cpp \ + scribus/scimagecachewriteaction.cpp \ scribus/scimagestructs.cpp \ scribus/scimgdataloader.cpp \ scribus/scimgdataloader_gimp.cpp \ @@ -1125,6 +1141,7 @@ scribus/scimgdataloader_tiff.cpp \ scribus/scimgdataloader_wpg.cpp \ scribus/sclayer.cpp \ + scribus/sclockedfile.cpp \ scribus/scmimedata.cpp \ scribus/scpageoutput.cpp \ scribus/scpageoutput_ps2.cpp \ @@ -1187,17 +1204,19 @@ scribus/util_text.cpp \ scribus/vgradient.cpp \ scribus/vgradientex.cpp \ - scribus/colormngt/sccolormngtengine.cpp \ - scribus/colormngt/sccolormngtenginefactory.cpp \ - scribus/colormngt/sccolormngtimplelem.cpp \ - scribus/colormngt/sccolormngtstructs.cpp \ - scribus/colormngt/sccolorprofile.cpp \ - scribus/colormngt/sccolorprofilecache.cpp \ - scribus/colormngt/sccolortransform.cpp \ - scribus/colormngt/sccolortransformpool.cpp \ - scribus/colormngt/sclcmscolormngtengineimpl.cpp \ - scribus/colormngt/sclcmscolorprofileimpl.cpp \ - scribus/colormngt/sclcmscolortransformimpl.cpp \ + scribus/colormgmt/sccolormgmtengine.cpp \ + scribus/colormgmt/sccolormgmtenginefactory.cpp \ + scribus/colormgmt/sccolormgmtimplelem.cpp \ + scribus/colormgmt/sccolormgmtstructs.cpp \ + scribus/colormgmt/sccolorprofile.cpp \ + scribus/colormgmt/sccolorprofilecache.cpp \ + scribus/colormgmt/sccolorprofiledata.cpp \ + scribus/colormgmt/sccolorspacedata.cpp \ + scribus/colormgmt/sccolortransform.cpp \ + scribus/colormgmt/sccolortransformpool.cpp \ + scribus/colormgmt/sclcmscolormgmtengineimpl.cpp \ + scribus/colormgmt/sclcmscolorprofileimpl.cpp \ + scribus/colormgmt/sclcmscolortransformimpl.cpp \ scribus/desaxe/desaxe_test.cpp \ scribus/desaxe/digester.cpp \ scribus/desaxe/digester_parse.cpp \ @@ -1353,6 +1372,7 @@ scribus/ui/prefs_fonts.cpp \ scribus/ui/prefs_guides.cpp \ scribus/ui/prefs_hyphenator.cpp \ + scribus/ui/prefs_imagecache.cpp \ scribus/ui/prefs_itemtools.cpp \ scribus/ui/prefs_keyboardshortcuts.cpp \ scribus/ui/prefs_miscellaneous.cpp \ @@ -1383,6 +1403,7 @@ scribus/ui/scinputdialog.cpp \ scribus/ui/scmenu.cpp \ scribus/ui/scmessagebox.cpp \ + scribus/ui/scmwmenumanager.cpp \ scribus/ui/scprogressbar.cpp \ scribus/ui/scrapbookpalette.cpp \ scribus/ui/scresizecursor.cpp \ |
|
I think this is the solution for bug 7920 and 7169. Big THX for your patch, hope it will be integrated into 1.3.5 or 1.4 stable |
|
mhh : Committed! Thanks you very much! |
Date Modified | Username | Field | Change |
---|---|---|---|
2010-01-18 22:46 | mhx | New Issue | |
2010-01-18 22:46 | mhx | File Added: scribus-trunk-imagecache-14-svn.diff | |
2010-01-19 09:36 | aliB | Note Added: 0023108 | |
2010-01-19 21:05 | jghali | Note Added: 0023109 | |
2010-01-19 21:05 | jghali | Status | new => resolved |
2010-01-19 21:05 | jghali | Fixed in Version | => 1.5.0svn |
2010-01-19 21:05 | jghali | Resolution | open => fixed |
2010-01-19 21:05 | jghali | Assigned To | => jghali |
2010-01-19 21:34 | cbradney | Relationship added | related to 0007920 |
2010-01-19 21:35 | cbradney | Relationship added | related to 0007169 |
2010-01-20 20:07 | cbradney | Status | resolved => closed |
2011-01-09 22:25 |
|
Issue cloned: 0009661 | |
2011-01-09 22:25 |
|
Relationship added | related to 0009661 |
2015-09-17 20:10 | Kunda | Category | Graphics / Image Frames => Graphics/Img Frames |
2015-09-17 20:11 | Kunda | Category | Graphics/Img Frames => Graphics / Image Frames |