Release Notes for XWiki Enterprise 4.0

Last modified by Thomas Mortagne on 2023/10/13

We've found an important issue in this release with our automatic data migration when upgrading from a version earlier than 4.0. Specifically the migration may fail when migrating wikis having stats, see XWIKI-7976 and when migrating DBs that was migrated from MySQL MyISAM previously, see XWIKI-7985. In addition we've also realized that the migration process can be slow on large XWiki databases and we have speed up the process. All these issue are fixed in release 4.1.3.

Before upgrading to this version, be sure to have a working backup of your database(s).

This is the first release of the 4.x cycle (Roadmap). This release brings reduced document database id collision, a new LDAP administration UI and quite a few improvements and bug fixes on Extension Manager, annotations, App Within Minutes and user profiles.

New and Noteworthy (since XWiki Enterprise 3.5 version)

Here are notable new features and improvements. For the full list of issues (including statistics) see the JIRA Dashboard.

Extension Manager improvements

Extension Manager

We improved the way the list of dependencies of an extension is displayed. You can now see the status of each dependency. In the following image you can see for instance that the Commons IO dependency is in conflict with the version of Commons IO bundled with XWiki Enterprise.

EM-dependencies.png

The extension status is also displayed when you view the extension separately. For instance, if you click on the Commons IO dependency link to get more details, you'll see the incompatibility message displayed as below:

EM-status.png

We also improved the search bar to allow users to search in different types of extensions. See below the result for searching "html" in core extensions, i.e. extensions bundled with XWiki Enterprise.

EM-search.png

The install/uninstall plan and log are now displayed in a tab inside the details section of an extension. We plan to make the new Progress tab display the install/uninstall status in real time in the near future.

EM-progressTab.png

XWiki Repository

  • Added a button to update an imported extension
  • It's now possible to extend an Extension sheet with custom informations. For example http://extensions.xwiki.org adds "Bundled with" and "Compatibility" fields.

Annotations merged with Comments by default

Starting with version 4.0M2, annotations have been merged by default with comments. More specifically, this means:

  • You can now reply to an Annotation (threaded annotations)
  • The Annotations tab has been removed and annotations are now located in the Comments tab where you can see both annotations and comments
  • The default annotation class is now XWiki.XWikiComments instead of AnnotationCode.AnnotationClass. It also means we've modified the XWiki.XWikiComments class by adding new fields to support annotations (See Migration Notes below).

annotations-merged-with-comments-screenshot.png

App Within Minutes improvements

The actions pane on the application home page now provides a link to edit and delete the application. Of course you need to have the proper rights to be able to see the links.

AWM-appActions.png

We also added an Actions column to the applications live table to be able to manage applications directly from the App Within Minutes home page. Again, you need to have the proper rights to be able to see the actions.

AWM-appsLiveTable-actionsColumn.png

User profile customization

An admin can now configure which information is displayed in the user profile (the "Profile" tab of a user's profile page). A new section has been added in the Administration where an admin can configure which fields of the XWiki.XWikiUsers class to display. The admin still has to add the new fields to the class the old way, using the class editor for the page XWiki.XWikiUsers, but he no longer needs to write any code.

userProfileCustomizer-screenshot.png

User directory improvements and user customized view

The User Directory can now display, as columns, filterable and sortable properties of users, as opposed to the previous version where you just had a fixed column that displayed a composed name in the form "first_name last_name (profilePageID)".

In addition, each user can now customize his view over the User Directory by adding or removing columns, or resetting it to the default columns.

The default displayed columns are "first_name" and "last_name", and the unmodifiable columns (that will always be displayed as first columns) are "_avatar" and "doc.fullName".

customizableUserDirectory-customize.png

In the near future, we're planning to allow Admins to customize globally the User Directory for all users too.

ColorThemes gradients customizable from ColorTheme Wizard

In version 3.4 we deprecated the old default bundled ColorThemes (because of incompatible variable mix). In version 4.0 this is no longer the case, old default bundled ColorThemes (like Bordo, Nature, etc.) are now being compatible with the 3.4 Colibri Skin changes. We fixed this by adding new gradient ColorThemes variables. All these gradient ColorTheme variables are now changeble from the ColorTheme Wizard.

ColorThemeWizard.png

LDAP administration UI

An LDAP administration interface is now provided to make it easy to configure LDAP directly from the wiki (you had to edit the XWiki Configuration files before). See the module documentation for more information on this topic.

admin-ldap-ui.png

New IRC Bot Application

The IRC Bot Application has been completely rewritten from the grounds up for better maintainability, improved speed and more features as described in the Documentation for the new IRC Bot Application.

ircbotapplication.png

Miscellaneous

For developers

Extension Manager moved to commons

Everything not specifically related to XWiki is now part of xwiki-commons. That means that anyone can get all Extension Manager features (easy live JAR installation/uninstallation from remote repositories, dependency management, etc.) in any Java application.

Classloader module moved to commons

Mostly been moved as Extension Manager dependency but also useful on it own as a tool to dynamically add new URLs in a classloader at runtime.

New Job module

The code to execute a task in background and provide related progress and isolated log has been extracted from Extension Manager and is now in its own module.

See Job Module.

Component related improvements

See Job Component.

Role moved from Class to Type

In order to support parameterized role like DocumentReferenceResolver<String>, all the APIs of ComponentManager and Component descriptors have been changed to take Type instead of Class. All the old APIs are still available and working, but have been deprecated.

ComponentRole is deprecated

You should now use @Role instead of @ComponentRole. The difference is that when using @Role the generic parameters will be taken into account as part of the role while with @ComponentRole the role stays the Class of the component role so be careful when you do this change, it might break everything if you don't want to take into account parameters (that's why a new annotation has been introduced instead of changing the behavior of ComponentRole).

References resolver/serializer and Provider now support parameter type as part of the role

This mean that declaring or injecting a such component no longer require any hint unless you want to provide/inject a variant for the same type.

Here are some example. Instead of:

@Inject
@Named("reference/default")
private DocumentReferenceResolver<EntityReference> resolver

@Inject
@Named("reference/current")
private DocumentReferenceResolver<EntityReference> resolver

@Inject
@Named("sometypeprovider")
private Provider<SomeType> resolver

you will write

@Inject
private DocumentReferenceResolver<EntityReference> resolver

@Inject
@Named("current")
private DocumentReferenceResolver<EntityReference> resolver

@Inject
private Provider<SomeType> resolver

All reference resolver and serializer provide static variables with the Type for both String and EntityReference. For example you can write:

DocumentReferenceResolver<EntityReference> resolver = componentManager.getInstance(DocumentReferenceResolver.TYPE_REFERENCE, "current")

New Velocity tool to serialize Java objects to JSON format

We introduced a new Velocity tool to be able to serialize Java objects to the JSON format. Checkout the documentation. Here's an example that shows how to serialize a Map object:

{{velocity}}
#set($map = {'name': 'xwiki', 'values': [1, 2, 3], 'enabled': true})
$jsontool.serialize($map)
{{/velocity}}

The result is:

{"name":"xwiki","values":[1,2,3],"enabled":true}

We also refactored XWiki.LiveTableResultsMacros page to generate the live table JSON in memory using a Map, which is serialized in the end using the new JSON tool. This makes it easier to customize the live table JSON. For example this is how the JSON for the AppWithinMinutes applications live table is generated:

{{include document="XWiki.LiveTableResultsMacros" /}}

{{velocity}}
#if($xcontext.action == 'get' && "$!{request.outputSyntax}" == 'plain')
  $response.setContentType('application/json')
  #set($map = {})
  #gridresult_buildJSON("$!request.classname" $request.collist.split(',') $map)
  ## Change the URL and permission for the edit and delete actions:
  ## * edit action must trigger the AppWithinMinutes wizard.
  ## * delete action must delete the application space.
  #foreach($row in $map.get('rows'))
    #set($editQueryString = "appName=$escapetool.url($row.get('doc_space'))")
    #set($discard = $row.put('doc_edit_url', $xwiki.getURL('AppWithinMinutes.CreateApplication', 'view', $editQueryString)))
    #if($row.get('doc_hasdelete'))
      ## Deleting an application requires space administration rights.
      #set($discard = $row.put('doc_hasdelete', $xwiki.hasAccessLevel('admin', $xcontext.user, "${row.get('doc_space')}.WebPreferences")))
    #end
    #set($rowDocRef = $services.model.createDocumentReference($row.get('doc_wiki'), $row.get('doc_space'), $row.get('doc_name')))
    #set($discard = $row.put('doc_delete_url', $xwiki.getURL($rowDocRef, 'deletespace', '')))
  #end
  $jsontool.serialize($map)
#end
{{/velocity}}

See Velocity Module.

Class reference always local

The class reference is now stored as a local reference in the BaseObject/BaseCollection which means that like in the storage you can't assign a class coming from another wiki to an xobject which was causing some issue when copying wikis. You can still provide absolute reference but the wiki reference will be automatically removed.

New Rights Implementation (Experimental)

A new experimental security authorization module is now available and is aimed to replace the old right service API during the 4.x cycle. This new module provide its public API from the org.xwiki.security.authorization.AuthorizationManager component. The whole authorization API has been rewritten from scratch with the following goals in mind:

  • better performance by providing a SecurityCache to cache both security rules and security access decisions.
  • allow extending the list of Right by registering new rights with a declarative description.
  • allow customising the way the authorization decision are made, by providing a custom AuthorizationSettler component.
  • stay independent from oldcore and use proper interfaces to made the authorization process as much as possible independent from the rest.

A bridge between this new security authorization module and the old implementation is also provided in org.xwiki.security.authorization.internal.XWikiCachingRightService which could be used in xwiki.cfg to enable this new authorization manager as the default one. The new module could also be used side by side with the old implementation.

The default implementation of the right settler, being written based on a fully restructured way to evaluate rights declaratively, has some subtle differences in the way security rules (right Xobjects) are interpreted and authorization decisions are made. Apart from improved consistency and yet to be discovered issues, the major differences still needing improvements are:

  • on an empty database, the guest user does not receive admin rights, and you need to enable and use superadmin to install an initial XAR.
  • in a multiwiki setup, user groups evaluated during decision are limited to those coming from the user wiki and those of the wiki of the targeted entity are not taken into account.

The full documentation of this new module, in particular the way it now evaluate rights declaratively and allow augmenting and customising authorization decision is yet to be written. The new module provide a large Javadoc as a reference in the meantime. For the big picture, the new right service is now named AuthorizationManager. Internally, it manipulates SecurityReference (as well as UserSecurityReference and GroupSecurityReference, to represent entities, user and group), SecurityRule (representing a right object) and SecurityAccess (representing an access level in the old nomenclature), which are stored in a SecurityCache using SecurityRuleEntry (a set of rules) and SecurityAccessEntry (the access of a given user). The AuthorizationManager delegate cache management to a SecurityCacheLoader which loads rules using a SecurityRuleLoader; and delegate itself the access decision for a given user and a set of rules to an AuthorizationSettler. This last one could be overridden to provide specific decision that could not be done in declarative mode.

New technical content hiding system

In this release we have only implemented the content hiding mechanism. At the moment, using it will not have any effect on the default UI you see (no documents will be filtered). We will gradually update all places listing documents in the UI to use the new mechanism in XWiki Enterprise 4.1. That said, if you're a developer we encourage you to test out the new content filtering API.

The basic support for hidden documents introduced in XWiki Enterprise 1.7 has been reimplemented differently (the previous implementation was not allowing anyone, including Admins, to see hidden documents!). The new logic is currently as follows (it'll be discussed and probably modified in XWiki Enterprise 4.1):

  • Simple users never see hidden documents
  • For advanced users, it is now possible to specify that a document should be hidden when editing the document (only wiki or wysiwyg edit modes):

    hidden.png

    And now there's a new setting in the user profiles, letting users chose if they want hidden documents to appear in search results or not:

    hiddenSettings.png

    As already mentioned above in warning, setting the Hidden configuration value will currently not have any effect on the default XWiki Enterprise pages since they're not calling the new filtering API yet (you'll need to wait for 4.1 for taht!). However it's supposed to work like this:

    • If the user doesn't set explicitly the Hidden setting value in his profile, he won't see hidden documents by default. Thus seeing hidden documents is an explicit choice.

To test this feature do the following:

  • Log as Admin on your wiki (or any user having edit rights)
  • Create a Page, say in Sandbox.HiddenFeatureTest and put the following content in wiki edition mode:
    {{velocity}}
    #set ($items = $services.query.xwql("where doc.name like 'S%'").setFilter("hidden").execute())
    There are $items.size() documents starting with 'S'

    #foreach ($item in $items)
    * [[$item]]
    #end
    {{/velocity}}
  • It'll print the number of matching documents (31 in a XWiki Enterprise 4.0 RC 1 with the default XAR) and the list of matching documents
  • Then go to Main.Search (for example), edit the page and check the Hidden checkbox and save
  • Go back to the Sandbox.HiddenFeatureTest  page and verify there's one less document printed now (it should say 30 in a XWiki Enterprise 4.0 RC 1 with the default XAR)
  • Then go to your user profile and select "Display Hidden Documents" to "Yes"
  • Go back to the Sandbox.HiddenFeatureTest  page and verify it's listing all documents again (it should say 31 in a XWiki Enterprise 4.0 RC 1 with the default XAR)

Miscellaneous

  • A new event ApplicationReadyEvent is sent when the Application is ready to service requests. Event Listeners can now listen to this event.
  • We've added a new Commons Text Module to contain XWiki-specific APIs related to text.
  • We added some new ColorThemes variables: $theme.fieldGradientColor, $theme.buttonPrimaryGradientColor, $theme.buttonSecondaryGradientColor, $theme.panelHeaderGradientColor, $theme.panelCollapsedGradientColor, $theme.menuGradientColor, $theme.menuContentBackgroundColor, $theme.menuContentLinkColor, $theme.menuContentGradientColor, $theme.submenuContentBackgroundColor

Upgrades

The following dependencies have been upgraded:

  • commons-codec 1.6
  • commons-configuration 1.8
  • commons-io 2.1
  • commons-net 3.1
  • HyperSQL 2.2.8
  • Infinispan 5.1.1
  • Junit 4.10
  • Maven Clover plugin 3.1.4
  • Selenium 2.20
  • Json-lib 2.4

Tested Browsers

Here's the list of browsers tested with this version (i.e. browsers that we've tested as working - Check the list of supported browsers):

Firefox
12
Firefox
3.6
Chrome
18
Internet Explorer
8
Internet Explorer
9

Tested Databases

HyperSQL
2.2.8
MySQL
5.x
Oracle
11.x
PostgreSQL
9.x

Known issues

Test Report

You can check the manual test report to learn about what was tested and the results on various browsers.

Backward Compatibility and Migration Notes

General Notes

If you're running in a multiwiki setup you'll also need to define the property xwiki.store.migration.databases in your xwiki.cfg file if you want to explicitly name some databases to be migrated as the default is now to migrate all databases. Database that are not migrated could not be accessed.

You may also want to import the default wiki XAR in order to benefit from all the improvements listed above.

Always make sure you compare your xwiki.cfg and xwiki.properties files with the newest version since some configuration parameters were added. Note that you should add xwiki.store.migration=1 so that XWiki will attempt to automatically migrate your current database to the new schema. Make sure you backup your Database before doing anything.

Migration of database IDs

You need to enable database migration (xwiki.store.migration=1) to upgrade existing databases to the new identifier format.

This is a major migration affecting almost all rows and relations in the database. I may takes a while before the wiki respond to your initial request when you are migrating a large database. The initial request may even timeout but the database migration continues in the background.

Avoid interrupting the database migration process, you can check the current progress of the database migration in the console output or the log file. The progress may be slow, so be patient during document id and object id conversion.

In case of interruption, the migration procedure could be restarted by restarting XWiki, and it will continue from where it has been interrupted.

We have taken many precautions to provide a reliable migration procedure, but due to the large changes it made, and this release being an unstable one, you should be really careful and ensure that you keep an appropriate backup until you are sure that your data has been properly migrated.

If you have custom mapped classes used externally, be aware that all internal identifier are now full 64bits longs.

Annotations Application

After an upgrade, in order to switch to the new annotations-merged-with-comments mode, you will have to update your settings. To do this, go to wiki Administration and:

  • change the annotation class in Applications > Annotations > Annotation type settings > XWiki class defining annotations structure from the old default AnnotationCode.AnnotationClass to the new default XWiki.XWikiComments
  • disable the Annotations tab by setting Look and Feel > Page elements > Document metadata visibility > Show document annotations to No

If you were/are using a custom annotations class, other than the previously default AnnotationsCode.AnnotationsClass, in the annotations configuration then the merging-with-comments feature will not be available to you. What you can do is to enable back the Annotations tab by going to the Administration > Look and Feel > Page Elements and changing the value of "Show document annotations " from "No" to "Yes". Now you will still be able to use your custom annotations without being affected by the changes.

Also, a migration script will automatically convert all your existing default annotations (that are using the default AnnotationCode.AnnotationClass) to the new default class (XWiki.XWikiComments). After the migration, the new database version will be 40001.

User profile customization

If you have a customized user profile tab (you have edited the "XWiki.XWikiUserProfileSheet" page), pay attention when importing the default XAR. If you want keep your customization and don't use the new User Profile Customization UI, just skip the "XWiki.XW

API Breakages

The following APIs were modified since version 3.5:

org.xwiki.bridge.AttachmentName: Class org.xwiki.bridge.AttachmentName removed
org.xwiki.bridge.DocumentAccessBridge: Method 'public java.io.InputStream getAttachmentContent(org.xwiki.bridge.AttachmentName)' has been removed
org.xwiki.bridge.DocumentAccessBridge: Method 'public java.lang.String getAttachmentURL(org.xwiki.bridge.AttachmentName, boolean)' has been removed
org.xwiki.bridge.DocumentAccessBridge: Method 'public java.util.List getAttachmentURLs(org.xwiki.bridge.DocumentName, boolean)' has been removed
org.xwiki.bridge.DocumentAccessBridge: Method 'public java.util.List getAttachments(org.xwiki.bridge.DocumentName)' has been removed
org.xwiki.bridge.DocumentAccessBridge: Method 'public org.xwiki.bridge.DocumentName getCurrentDocumentName()' has been removed
org.xwiki.bridge.DocumentAccessBridge: Method 'public org.xwiki.model.reference.DocumentReference getCurrentUserReference()' has been added to an interface
org.xwiki.bridge.DocumentAccessBridge: Method 'public org.xwiki.bridge.DocumentModelBridge getDocument(org.xwiki.bridge.DocumentName)' has been removed
org.xwiki.bridge.DocumentAccessBridge: Method 'public org.xwiki.bridge.DocumentName getDocumentName(java.lang.String)' has been removed
org.xwiki.bridge.DocumentModelBridge: Method 'public org.xwiki.bridge.DocumentName getDocumentName()' has been removed
org.xwiki.bridge.DocumentName: Class org.xwiki.bridge.DocumentName removed
org.xwiki.gwt.user.client.ui.ListBox: Removed com.google.gwt.event.dom.client.KeyDownHandler from the set of implemented interfaces
org.xwiki.gwt.user.client.ui.ListBox: Removed com.google.gwt.event.dom.client.KeyPressHandler from the set of implemented interfaces
org.xwiki.gwt.user.client.ui.ListBox: Removed com.google.gwt.event.dom.client.KeyUpHandler from the set of implemented interfaces
org.xwiki.gwt.user.client.ui.ListBox: Method 'public void onKeyDown(com.google.gwt.event.dom.client.KeyDownEvent)' has been removed
org.xwiki.gwt.user.client.ui.ListBox: Method 'public void onKeyPress(com.google.gwt.event.dom.client.KeyPressEvent)' has been removed
org.xwiki.gwt.user.client.ui.ListBox: Method 'public void onKeyUp(com.google.gwt.event.dom.client.KeyUpEvent)' has been removed
org.xwiki.query.Query: Method 'public org.xwiki.query.QueryFilter getFilter()' has been added to an interface
org.xwiki.query.Query: Method 'public org.xwiki.query.Query setFilter(org.xwiki.query.QueryFilter)' has been added to an interface
org.xwiki.gwt.wysiwyg.client.plugin.line.IELinePlugin: Removed com.google.gwt.event.dom.client.KeyDownHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.IELinePlugin: Removed com.google.gwt.event.dom.client.KeyPressHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.IELinePlugin: Removed com.google.gwt.event.dom.client.KeyUpHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.IELinePlugin: Removed com.google.gwt.event.shared.EventHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.LinePlugin: Removed com.google.gwt.event.dom.client.KeyDownHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.LinePlugin: Removed com.google.gwt.event.dom.client.KeyPressHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.LinePlugin: Removed com.google.gwt.event.dom.client.KeyUpHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.LinePlugin: Removed com.google.gwt.event.shared.EventHandler from the set of implemented interfaces
org.xwiki.gwt.wysiwyg.client.plugin.line.LinePlugin: Method 'public void onKeyDown(com.google.gwt.event.dom.client.KeyDownEvent)' has been removed
org.xwiki.gwt.wysiwyg.client.plugin.line.LinePlugin: Method 'public void onKeyPress(com.google.gwt.event.dom.client.KeyPressEvent)' has been removed
org.xwiki.gwt.wysiwyg.client.plugin.line.LinePlugin: Method 'public void onKeyUp(com.google.gwt.event.dom.client.KeyUpEvent)' has been removed
org.xwiki.gwt.wysiwyg.client.plugin.list.ListBehaviorAdjuster: In method 'public ListBehaviorAdjuster()' the number of arguments has changed
org.xwiki.gwt.wysiwyg.client.plugin.list.ListBehaviorAdjuster: Method 'protected void dispatchKey(com.google.gwt.user.client.ui.Widget, int, org.xwiki.gwt.dom.client.Event)' has been removed
org.xwiki.gwt.wysiwyg.client.plugin.list.ListBehaviorAdjuster: Method 'public org.xwiki.gwt.user.client.ui.rta.RichTextArea getTextArea()' has been removed
org.xwiki.gwt.wysiwyg.client.plugin.list.ListBehaviorAdjuster: Method 'public void setTextArea(org.xwiki.gwt.user.client.ui.rta.RichTextArea)' has been removed
org.xwiki.gwt.wysiwyg.client.plugin.macro.exec.InsertExecutable: In method 'public InsertExecutable(org.xwiki.gwt.wysiwyg.client.plugin.macro.MacroSelector)' the number of arguments has changed
org.xwiki.wysiwyg.server.WysiwygEditorConfiguration: Method 'public java.lang.Integer getHistorySize()' has been added to an interface

Get Connected