Recently, I came across an interesting requirement. The requirement was as below:
- The scheduler should be implemented in the clustered environment which pulls content from an external repository.
- There should be separate node for the scheduler operation and it should not serve the users.
- The scheduler should run on a specific time e.g. every midnight.
- An admin should also be able to define a job to pull content from external source. This job should also run on the same node as the scheduler.
There were obvious reasons behind having these requirements. The performance of the nodes serving users should not be affected due to batch job (scheduled job). Also, if anything goes wrong in the batch job, we should be able to bounce the node back without affecting the user sessions.
I proposed following system architecture for the implementation of the same.
Here, the Web Servers are connected to App Servers 3 to 6. These servers will serve user request which comes from Web Server 1 and 2. The App Server 1 & 2 will have the scheduler portlet deployed. This design solves the requirement of having separate nodes for scheduler and serving users.
Now, for the actual implementation again I have two different requirements to cater.
- I have to have a scheduler which runs every midnight to fetch external content
- I need to have a mechanism where an admin can set a job manually to fetch the content
This requirement can be fulfilled with the use of two portlets. One portlet say Fetch Content Portlet will fetch the content based on the schedule and other portlet say Set Schedule Portlet will provide a facility to the admin to set up manual job to fetch content.
The second portlet is pretty simple Liferay Portlet. It will deployed on Node 3 to Node 6 and available in Control Panel. An admin will be able to use this portlet to add an entry to a database table to fetch the content.
The first portlet is an interesting one. It will need two different schedulers - 1) To run the job every midnight to fetch content 2) To check the DB table every 10 minutes to see if an entry has been added by admin to fetch the content.
The implementation is as below:
Add following entry into your Fetch Content Portlet's liferay-portlet.xml file:
<scheduler-entry>
<scheduler-description>fetch-video</scheduler-description>
<scheduler-event-listener-class>
com.test.scheduler.FetchVideoJob
</scheduler-event-listener-class>
<trigger>
<cron>
<cron-trigger-value>0 0 0 1/1 * ?
*</cron-trigger-value> // This will run the cron job every midnight
</cron>
</trigger>
</scheduler-entry>
<scheduler-entry>
<scheduler-description>check-schedule-entry</scheduler-description>
<scheduler-event-listener-class>
com.test.scheduler.CheckScheduleEntryJob
</scheduler-event-listener-class>
<trigger>
<cron>
<cron-trigger-value>0 0/10 * 1/1 * ?
*</cron-trigger-value> // This will run the cron job every 10 minutes
</cron>
</trigger>
</scheduler-entry>
The first entry will be for the job to run every midnight. Second entry is for the scheduler which will ping the DB every 10 minutes to check for a job entry. If an entry is found, it will fetch the content. And once the operation is completed, it will invalidate (or delete) the entry.
We also need to have scheduler classes for the each scheduler. They are as below:
FetchVideoJob – To fetch the videos every midnight.
package com.test.scheduler;
import com.liferay.portal.kernel.messaging.Message;
import
com.liferay.portal.kernel.messaging.MessageListener;
import
com.liferay.portal.kernel.messaging.MessageListenerException;
import
com.liferay.portal.kernel.log.Log;
import
com.liferay.portal.kernel.log.LogFactoryUtil;
public class FetchVideoJob
implements MessageListener
{
private static final Log LOGGER
= LogFactoryUtil.getLog(FetchVideoJob.class);
public
void receive(Message arg0) throws MessageListenerException {
//Write logic to fetch the content from the external repository. This
scheduler //will run on every 12.00 am.
}
}
CheckScheduleEntryJob
– To check an entry in the DB every 10 minutes. If an entry is found, fetch the
videos and invalidate (or delete) the entry.
package com.test.scheduler;
import
com.liferay.portal.kernel.messaging.Message;
import
com.liferay.portal.kernel.messaging.MessageListener;
import
com.liferay.portal.kernel.messaging.MessageListenerException;
import com.liferay.portal.kernel.log.Log;
import
com.liferay.portal.kernel.log.LogFactoryUtil;
public class CheckScheduleEntryJob
implements MessageListener
{
private static final Log LOGGER
= LogFactoryUtil.getLog(CheckScheduleEntryJob.class);
public
void receive(Message arg0) throws MessageListenerException {
// Write logic to check the DB table if there is any manual entry. If
there is any //manual entry, it should fetch the content from external
repository.
// Once the content
are fetched & stored into LR DB, invalidate the entry in the //DB table.
}
}
We can even change the logic in such a way that an admin will set a time for the manual job to be executed next. In this case, we have to take the time an admin has entered and compare with the current system time to determine if the job needs to be executed or not.
