Implemented in 4.1
Official Documentation Available
This topic is now covered in Imaging module.
- Images will be generated on demand (not on save as in magnolia-module-image-filtering-1.x)
- Generated images can be reused (i.e do not store 2907 copies of the same menu item image)
- Avoid passing parameters in url to avoid hacks
- URLs should be kept relatively short and easy to generate
- Store/cache rendered content in a separate workspace. Solve issues related to cache, etc. (generated image needs to be in synch with their "source")
- Performance and scalability are issues we've solved with a few tricks in a few different places in customer projects; consolidate these solutions in this module
Features using this module could be:
- Image galleries (thumbnail creation) : resize/crop
- Banner generator (2.24)
- Teasers, ...
- Designer configures filters or dialogs to generate desired images (grayscale, resized, added border and text overlay) so that authors only have the minimal amount of options. (upload or select a background image, enter the text to overlay)
Sample use cases
"N days left to register to Magnolia conference"
- Specific Text ImageOperation: can calculate N based on a configured date, prints it. (extend
- Doesn't need a specific ParameterProvider
- Background can be either image specified through Parameter or configured
- generated-images-cache: generated image node could be removed through a scheduled task (or cache could be disabled, relying solely on page-level cache)
- page-level cache: scheduled task could also flush the specific url every day.
- Specific ParameterProvider (or something akin to the StringParameterProvider) - N is basically passed through the url
- Doesn't need a specific ImageOperation (or a simple PassedText if I don't commit it - it's currently not in the codebase)
- Background image can be either configured or specified through Parameter if further specializing the ParameterProvider
- No concerns about generated-images-cache nor page-level cache.
info.magnolia.module.imagefiltering.operations.ImageOperation interface is a simple interface to manipulate images. We provide a wrapper around
java.awt.image.BufferedImageOp, so any existing implementation of this interface should be reusable. (see link to the
ImageOperationChain is an implementation of
ImageFilter that delegates to a list of other
ImageOperation s. By making it the default implementation for c2b and using the
CollectionPropertyHidingTransformer, we can achieve a readable configuration of chained filters.
Working "on" an image (i.e cropping an existing image) is also be an
ImageOperation, i.e the first in the chain, which will load the image.
info.magnolia.imaging.ImageGenerator interface is the entry point to image generation. The default implementation is also
Image generation parameters can come from various places:
- in the ImageOperation configuration itself
- in the content node we're generating an image for (example: crop coordinates, piece of text, ...)
- in another node ? (example: navigation menu item text)
- in the URL ? (path or parameters) (if the template displaying the image determines some of the parameters/variations to use)
ParameterProvider and ParameterProviderFactory
ParameterProvider encapsulates a parameter. One single
ParameterProvider is passed to an
ImageGenerator. That doesn't mean there is only a single parameterizable option or item for the whole generated image, because the Parameter itself can be any type of object (from a String to a Content node to an HashMap).
ParameterProvider implementations are typed, bound to the type of parameter they encapsulate. (ie
ParameterProvider<Content> wraps a
ImageOperation implementations can be written such that they ignore the Parameter type, or if they don't, they need to be written against a specific type of
ParameterProvider (using generics). This makes them "stronger" (less guess-work when it comes to getting parameters). If operations are well separated (i.e do one single thing), it is fairly easy to come up with multiple implementation of a similar operation that uses different types or Parameters (see
info.magnolia.imaging.ParameterProviderFactory couple is meant to abstract away where and how the parameters are passed to an
ParameterProviderFactory implementation are written such that they can instanciate a specific type of
ParameterProvider (existing implementations of
ParameterProvider are only typed implementation of the interface) for a given "environment" (a servlet request, a workitem, ...)
Parameter vs Configuration
It is important to differentiate parameter and configuration. While the whole image generation is dynamic, the parameter is the only "attribute" that will make different requests generate different images. I think as little as possible should be parameterized, and as much as possible should be configured.
Take for example the
info.magnolia.imaging.operations.text.TextFromNode operation: the text that is added to the image is not configured nor is it a parameter. The parameter is the "location" of the text (the node), and the configuration determines which property of the given node to use for the text.
The same could be applied for a cropping operation: the source would also be a node, and the configuration would tell us which property(ies) of that node store the CropInfo.
The configuration might seem a little redundant with, for instance, dialogs configuration (because the dialog itself will also have info on how/where to store the CropInfo), but it also helps having simpler URLs (see above); we also stay relatively strongly-typed (by avoiding a "big pot" of parameters in the form of a HashMap).
- Lots of TODOs, see code.
- Absolutely no caching, threading, etc. This should be hidden/abstracted away from the components that exists at the moment. (separation of concerns)
- Http headers need to be set appropriately.
Additional filters to provide as examples:
- "Default" cropping, i.e cropping that works without a UI, cropping properly (centered) to the configured ratio (i.e of the target size)
- Border (adds configurable borders around pictures)
- Watermark (adds a static piece of text in a corner of the picture)
- AuthorInfo (adds a dynamic piece of text in a corner of the picture)
- TextForNavigation ? (i.e uses navTitle as dynamic text..?)
Having a repeatable control (upload n images, upload+crop n images, ...) would be helpful but not something that should implemented in this specific module. MME-29@jira
Try it out!
- Build and deploy the
magnolia-module-imagingmodule. Other modules not needed at the moment. (crop-ui does nothing afaik, imaging-tools just adds the page that displays the available ImageIO providers)
temp-bootstrapfolder in the appropriate location.
(you'll probably need to restart since the imaging module had no config node when you started up)
The current setup/examples all use the one servlet mapped to
/.imaging/*, the first pathInfo element determines the
ImageGenerator name, the second the workspace and the rest is the path of the node to load. This is handled by
ImageServlet (generator name) and
ParameterProviderFactory which loads a node based on the request uri).
- Create a page called "test" in the website repo
- Go to http://localhost:8080/magnolia-empty-webapp-4.1-SNAPSHOT/.imaging/sample-chain1/website/test.jpg
- This uses an image background loaded off the web (which is why it's slow). ("website/test" must exist because that's how the current ProviderParameterFactory behaves)
- It also resizes the image and adds a fixed text to it.
- With STK, create a dummy "Section" page under "test" called "hello", and in there an "Article" page called "world".
- In this new page, upload a teaser image. Also type in something for the "abstract".
- Go to http://localhost:8080/magnolia-empty-webapp-4.1-SNAPSHOT/.imaging/sample-chain2/website/test/hello/world.jpg
- This now uses an image loaded off the path "website/test"; the propertyName of the binary is configured in the ImageOperation (/generators/sample-chain2/operations/img)
- It also resizes the image and adds to texts 1) a fixed text 2) the "abstract" text from the "test/hello" node (this property name is also configured in the ImageOperation)
- Create a folder in dms called "myFolder" (in the URL column, i.e the node name). Create a document in there ("myDoc" - in the URL column, i.e the node name) and upload an image to it.
- Go to the generator configuration (
/generators/sample-chain2/operations/) - change the
- Change the
- I suppose that by you've guessed that you should go to http://localhost:8080/magnolia-empty-webapp-4.1-SNAPSHOT/.imaging/sample-chain2/dms/myFolder/myDoc.jpg, and that what this does is fairly obvious
wood chain too
We had to solve a bunch of quite complex problems in the past, specifically with the Seat project. Have a look at http://seat.de/seat_/magazin.html or see the attached screenshot. There are a bunch of generated images on that page:
- top image - left (given background photo, text, curve shape "cutting" in the original photo)
- menu items on the top right (text, generated background from #1, which can include part of the photo)
- menu items on the bottom right (similar, usually no photo)
- menu items on the bottom left (simple text, plain background; colors depend on the "context", depending on the page, the background might be black, white, ..)
Issues to solve:
- Avoid generating N copies of the same image.
- Get various parameters, encode/decode URLs, ...
- Performance: with so many images to generate, it was necessary to queue and pool the image generation jobs.
- Dependencies/locking: since #2 and #3 depend on #1, we needed some way to ensure that #1 was generated first, no matter what order the http requests came in
- Instance synchronization: TODO.
Discussion about parameters
Considering most images are generated "for" a given node, or based on a given node, (which can be an image, a piece of text...), we could assume the following URL schema:
http://localhost:8080/magnoliaPublic/image-generator/<ImageGenerator name>/<Parameters> \_>contextPath \_>Servlet \_> workspace and path to node
Parameters could be further decomposed into:
This is assuming that ImageOperation implementations know how to get the parameters they need, either through the given node (i.e they now that the cropper configuration is stored as the property or subnode "mgnl:cropperInfo"), or through their own configuration.
Parameters decoding can be implemented, for instance where the path elements would be, respectively, the locale, the bundle name (or path) and key to a text message:
/de_CH/info/magnolia/module/templatingkit/messages/link.readon.gif (or rather, with a configured filter,
de_CH/link.readon.gif, where the bundle name would be configured at
If we have such strict URL schemes:
- it is fairly easy to build urls
- it is fairly easy understand the mechanisms involved, while still protecting against malicious parameters (although the weak point would be the node path)
- we can store generated images under the same path in the
generated-imagesworkspace. Path to the generated image node would simply be
Problems / other approaches
Using AggregationState causes issues because the AggregationFilter checks if the path is a page node or a binary (dms nodes are neither, for instance)
See Proposal - merging of URI2RepositoryMapping and VirtualURIMappings
See Proposal - overhaul and refactoring of AggregationState
We could have a system that generates a unique ID for image/set of parameters. The parameters are then stored in the
generated-images workspace. The generated uuid is not the jcr:uuid.
- this ensure uniqueness: one ID = one image. If the parameters/sources change, the ID will be different. This is good for caches, proxies, ...
- complete safety (nothing can be tempered with in the uri)
- mechanisms involved are a little more obscure to understand: URLs are more complicated, a helper class HAS TO be used to generate image links (whereas with the proposed mechanism, templaters could write their links "manually"), ...
- templates (through the use the link building class) are more tightly coupled with the ImageFilters (even though there are already IS coupling since the template has to know the filter name and what the
<ImageFilterParameters>URL portion should be (but this should usually be simple and straighforward)
In any case, there needs to be some guarantee that the content upon which the image generation is based has not changed; if a new background image has been uploaded, or text changed, the generated images needs to be regenerated.
Generating links and images
If we have a helper class that generates uuid-based links (see above), then why not trigger the image generation already at that moment? This was probably the first thing that was removed from the Seat implementation, but considering the following, wouldn't it be an option now ?
- Image generation is threaded; happens in separate thread(s) (or a pool of)
- Image generation is blocking: if an http request comes in, requesting image #152, it is blocked until the image is available (alternatively, it could be sent with a "404" or other temporary image - possibly after a timeout)
- We use java.util.concurrency for managing threads and tasks, so it could be fairly clean too.
I have the feeling none of these approaches is essentially contradicting.
- Strict URL schemes: the most straightforward. Image is generated upon request. (if needed)
- Generating links and images: to be validated. Image generation task is launched upon link generation.
- UUIDs with deferred generation: Link generation stores the image "job" (definition of the job, parameters), but the image is still generated upon request (if needed)
- Generic filters: http://www.jhlabs.com/ip/filters/index.htmlprovides a nice api which we could reuse for all "effects" type of filters (grayscale, colors, blur, etc) - pretty trivial as all filters have an empty constructor and setters for their specific parameters. (this is what I used for my grayscale p.o.c)
- jhlabs/filters has been in the process of moving to https://pixels.dev.java.net/ since 2007; I had an email from the author regarding this at the time, not sure what the current status is, as his own webpage has not been updated to reflect this yet.
- I will actually help directly on this project and have a complete new release (the current one in the maven central repo is outdated, and doesn't correspond to the sources delivered on the homepage)
- There is still a lot to learn: Using Java 2D's Image-Processing Model: http://www.informit.com/articles/article.aspx?p=1013851
This (http://www.jhlabs.com/ip/filters/index.html) is indeed a very cool resource. Can you clarify the license? I'd love to see the http://www.jhlabs.com/ip/filters/MirrorFilter.html applied in a new coverflow module.
The license is ASL: http://www.jhlabs.com/ip/filters/download.html
Most excellent, thanks Greg.
From my perspective, the repeatable control is less important then supporting the selection of a DMS folder instead. FYI, the latter is already implemented in Sitedesigner.
re: dms selection, the selection is feasible since a while, the feature mentioned here (MGNLIMG-15@jira) is the processing/filtering of one node. It's likely to be fairly simple to implement, but the structure of a node in dms is slightly different than a binary attached to a paragraph.
Selecting and processing a complete folder is yet another issue.
Ordering/importance of "features to be implemented" is indeed approximate