problem front-end catalog display after install (among others)

Added by Kazmierski Mathieu over 2 years ago

Hi,

So as to test EasySDI solutions,
I've just set up a local fresh install of :
LAMP
Joomla3.3
EasySDI 4.3.1
Geoserver 2.6.4
Geonetwork 2.6.3
on
a VM with Ubuntu Server 14.04 LTS.

I went through the install process successfully with each module (joomla, geonetwork, easysdi, proxy, monitor), according to the tests you described in the documentation. (I would add that I'm not really sure what i've done with the xsl file, I copied it in ../joomla/media/easysdi/catalog/xsl/ folder. It has a positive effect on the xslt list when configuring EasySDI>Catalog, but that's all)
Then, I've configured the catalog service in easysdi backend : physical service, virtual service & access rules, and add the the catalog item to the front-end menu.

First question : the front-end displays only the "search" button (look at capture.png), with nothing else around ; is that a default behaviour? I guess it's not...

Second question : when I try to search something, it gets worse and returns errors : first, a yellow "Warning : An error occured while trying to get catalog records" and eventually a "Error : Error in EasySDI Proxy. Consult the proxy log for more details."

So I did, to find out three types of suspicious messages :
  • before the error log, I looked after the query my front-end have sent to the server :
    http://localhost/joomla/index.php/catalogue?view=catalog&search=true&id=&preview=&jform%5Bsearchtype%5D=simple&jform%5Bid%5D=
    

    which seems to be very empty imo and very unlikely to return anything...problem of FORM i guess.
  • then, I checked the easysdi-proxy.log, which have sent the following SQL request which begins like this :
    09 juil. 2015 15:08:27,341 DEBUG SQL:104 - select session0_.session_id as session_1_66_, session0_.client_id as client_i2_66_, session0_.data as data3_66_, session0_.guest as guest4_66_, session0_.time as time5_66_, session0_.userid as userid6_66_, session0_.username as username7_66_ from jml_session session0_ where session0_.session_id=?
    ...
    09 juil. 2015 15:08:27,399 DEBUG SQL:104 - select sdiusers0_.user_id as user_id17_67_1_, sdiusers0_.id as id1_53_1_, sdiusers0_.id as id1_53_0_, sdiusers0_.access as access2_53_0_, sdiusers0_.asset_id as asset_id3_53_0_, sdiusers0_.catid as catid4_53_0_, sdiusers0_.checked_out as checked_5_53_0_, sdiusers0_.checked_out_time as checked_6_53_0_, sdiusers0_.created as created7_53_0_, sdiusers0_.created_by as created_8_53_0_, sdiusers0_.description as descript9_53_0_, sdiusers0_.guid as guid10_53_0_, sdiusers0_.modified as modifie11_53_0_, sdiusers0_.modified_by as modifie12_53_0_, sdiusers0_.notificationrequesttreatment as notific13_53_0_, sdiusers0_.ordering as orderin14_53_0_, sdiusers0_.params as params15_53_0_, sdiusers0_.state as state16_53_0_, sdiusers0_.user_id as user_id17_53_0_ from jml_sdi_user sdiusers0_ where  sdiusers0_.State = 1 and sdiusers0_.user_id=?
    ...etc...
    
  • third message,
    ...
    08 juil. 2015 11:17:32,264 WARN AbstractEhcacheRegionFactory:180 - HHH020003: Could not find a specific ehcache configuration for cache named [org.hibernate.cache.spi.UpdateTimestampsCache]; using defaults.
    08 juil. 2015 11:17:32,268 INFO StandardQueryCache:86 - HHH000248: Starting query cache at region: org.hibernate.cache.internal.StandardQueryCache
    08 juil. 2015 11:17:32,271 WARN AbstractEhcacheRegionFactory:180 - HHH020003: Could not find a specific ehcache configuration for cache named [org.hibernate.cache.internal.StandardQueryCache]; using defaults.
    08 juil. 2015 11:17:32,295 WARN AbstractEhcacheRegionFactory:180 - HHH020003: Could not find a specific ehcache configuration for cache named [org.easysdi.proxy.domain.SdiCswSpatialpolicy]; using defaults.
    08 juil. 2015 11:17:32,298 WARN EhcacheAccessStrategyFactoryImpl:57 - HHH020007: read-only cache configured for mutable entity [org.easysdi.proxy.domain.SdiCswSpatialpolicy]
    08 juil. 2015 11:17:32,439 WARN AbstractEhcacheRegionFactory:180 - HHH020003: Could not find a specific ehcache configuration for cache named [org.easysdi.proxy.domain.SdiWmtslayerPolicy]; using defaults.
    [...]
    

    ...and so on for other java classes. Though, I have configured the ehcache.xml as suggested, and gave tomcat user rights on the directory.

I've inspected other error logs...

The CSW log says :

09/07/2015 14:47:58:639 [http-8180-2] ERROR ProxyLogger:556 - net.sf.json.JSONException: JSONObject["infrastructureID"] not found.

So i tried what you described here : https://forge.easysdi.org/boards/34/topics/599
I've changed it but still no improvement.

I give you all the log I've found

I tried to do a GetRecordById behing the proxy, and it works perfectly fine...So I guess it somewhere between joomla, easysdi and the proxy.

Thanks for any help you could provide!
By the way, thanks for your documentation, really helpful to begin with easySDI!

MathieuK


Replies (15)

RE: problem front-end catalog display after install (among others) - Added by Blatti Yves over 2 years ago

Hi Mathieu !
I'm on holiday so it’ll be hard for to give you answers in a short time. I hope someone else will be able to do so (Thomas, do you hear me ? ;-))

For your questions:


Q0

I would add that I'm not really sure what i've done with the xsl file, I copied it in ../joomla/media/easysdi/catalog/xsl/ folder.

That the thing to do ! You can now customize this file for rendering the search results and the metadata sheet view. (But we'll try to fix your problems first...)


Q1

the front-end displays only the "search" button (look at capture.png), with nothing else around ; is that a default behaviour? I guess it's not...

It is the default behavior: you have to chose the fields you want to use in the catalog you created. In backend, in a catalog (easysdi/catalog/catalogs): you can choose in different default search fields (free text, dates, resources types etc…) and add custom CSW filters (they have to match in lucene indexed fields).


Q2

Second question : when I try to search something, it gets worse and returns errors : first, a yellow "Warning : An error occured while trying to get catalog records" and eventually a "Error : Error in EasySDI Proxy. Consult the proxy log for more details."

The "Error" comes from Java proxy, and the "Warning" is caused by this error in joomla component. So we have to focus on the first Error.
You have a problem :

+-------------------------+         +------------------+           +----------+
|easySDI Joomla components|---------|easySDI Java proxy|-----------|geonetwork|
+-------------------------+         +------------------+           +----------+
                               ^                             ^
                            here(1)         OR             here(2)

See "Q6" and "To check"


Q3

before the error log, I looked after the query my front-end have sent to the server :
http://localhost/joomla/index.php/catalogue?view=catalog&search=true&id=&preview=&jform%5Bsearchtype%5D=simple&jform%5Bid%5D=

The search query only contains your search fields... since you don't have any, it is correct.
The others filters to generate the CSW request (et by your catalog, the access scopes, resource types etc... ) are added serverside.


Q4

[...] easysdi-proxy.log, which have sent the following SQL request which begins like this :
09 juil. 2015 15:08:27,341 DEBUG SQL:104 - select session0_.session_id as session[...]

If you only have those debug lines, everything is OK (Database connections, environment)


Q5

third message [...] WARN AbstractEhcacheRegionFactory

Those are considered "normal" since we want Ehcache to use the defaults. (I don't know why it's a warning not an info)


Q6

The CSW log says :
09/07/2015 14:47:58:639 [http-8180-2] ERROR ProxyLogger:556 - net.sf.json.JSONException: JSONObject["infrastructureID"] not found.

(Note: I created an issue for that here #1060, I suppose this is an installer bug)

The CSW log will be your friend when you have catalog configuration issues, it is normally quite verbose ;-)

You say:

So i tried what you described here : https://forge.easysdi.org/boards/34/topics/599
I've changed it but still no improvement.

Do you mean that you have added an "infrastructureID" using this SQL snippet ?

update
jos_extensions
set params = CONCAT('{"infrastructureID":"',UUID(),'","descriptionlength":"","logowidth":"","logoheight":""}')
where name = 'com_easysdi_core'

  • Did you adapt the joomla table prefix (replaced the "jos_" by your joomla table prefix) ?
  • Can you see the new infrastructureID in your easySDI configuration now ?
  • Does your message still appears in CSW log ?
    • The proxy will fail to create a correct CSW request without it, if the problem is still present we'll have to focus on this

What to check ? :

I don't know extacly what you have done so far... I give you here the "classics":

  • Did you go through easySDI all components configuration to check that the required fields are set, and save the configuration again (just in case, to ensure that they are saved in joomla JSON cache)
  • In the catalog configuration, did you set the default language and encoding ?
  • You can have a look at my "sample_catalog_config.png", note that the catalog URL is the virtual service !
  • Do you have a metadata model ? (I suppose since you had issues on import : https://forge.easysdi.org/boards/34/topics/627)
  • Do you have resource types and profiles configured ?
  • In your geonetwork physical service
    • Is you URL correct and accessible by the proxy machine (sample_geonetwork_conf_tab1.png)
    • Are your provider and authentication URL correct with username+password for a geonetwork user that has transaction rights ? (sample_geonetwork_conf_tab2_provider.png)
    • I guess this service should be OK, otherwise you would have errors in the CSW log...
  • You geonetwork virtual service should not be a problem (you just have to set the correct physical service...)
  • In your policy for the geonetwork virtual service, can you check de from and to dates
  • In your catalog have you set at least one resource type and a CSW sorting field (typically title)

Last tip for today:
  • Clean your logs: they are very verbose and i like to clean them up, then run a single request to see what happens...

I hope this can help

Regards

Yves

RE: problem front-end catalog display after install (among others) - Added by Kazmierski Mathieu over 2 years ago

Hi Yves!
Thanks a lot for the answer (even more if you were on holiday...I really do appreciate)!
I tried to follow up your leads.
Here is my feedback :


Q0

 I would add that I'm not really sure what i've done with the xsl file, I copied it in ../joomla/media/easysdi/catalog/xsl/ folder.

That the thing to do ! You can now customize this file for rendering the search results and the metadata sheet view. (But we'll try to fix your problems first...)

OK, that's what I've done.


Q6

 The CSW log says :
 09/07/2015 14:47:58:639 [http-8180-2] ERROR ProxyLogger:556 - net.sf.json.JSONException: JSONObject["infrastructureID"] not found.

(Note: I created an issue for that here #1060, I suppose this is an installer bug)

The CSW log will be your friend when you have catalog configuration issues, it is normally quite verbose ;-)

You say:

 So i tried what you described here : https://forge.easysdi.org/boards/34/topics/599
 I've changed it but there's still no improvement.

Do you mean that you have added an "infrastructureID" using this SQL snippet ?

update
jos_extensions
set params = CONCAT('{"infrastructureID":"',UUID(),'","descriptionlength":"","logowidth":"","logoheight":""}')
where name = 'com_easysdi_core'

Yes, I did.
  • Did you adapt the joomla table prefix (replaced the "jos_" by your joomla table prefix) ?
    Yes, my install is jml_ prefixed for joomla tables, I'd noticed this variation in my config.
  • Can you see the new infrastructureID in your easySDI configuration now ?
    Yes.
  • Does your message still appears in CSW log ?
  • Nop, nothin on infrastructureID in the tomcat log.
  • Did you go through easySDI all components configuration to check that the required fields are set, and save the configuration again (just in case, to ensure that they are saved in joomla JSON cache)
    I think so.
  • In the catalog configuration, did you set the default language and encoding ?
    Yes.
  • You can have a look at my "sample_catalog_config.png", note that the catalog URL is the virtual service !
    Yes, it is.
  • Do you have a metadata model ? (I suppose since you had issues on import : https://forge.easysdi.org/boards/34/topics/627)
    Yes.
  • Do you have resource types and profiles configured ?
    Yes, according to the schema you'd provided in the tutorial.
In your geonetwork physical service,
  • Is you URL correct and accessible by the proxy machine (sample_geonetwork_conf_tab1.png)?
    It seems so, since I can query geonetwork through my virtual service. (http://192.168.1.82:8180/proxy/vgeonetwork?SERVICE=CSW&REQUEST=GetCapabilities)
  • Are your provider and authentication URL correct with username+password for a geonetwork user that has transaction rights ? (sample_geonetwork_conf_tab2_provider.png)
    Yes.
I guess this service should be OK, otherwise you would have errors in the CSW log...
You geonetwork virtual service should not be a problem (you just have to set the correct physical service...)
  • In your policy for the geonetwork virtual service, can you check de from and to dates
    It's OK, it's supposed to run 'til 2025 :-)
  • In your catalog have you set at least one resource type and a CSW sorting field (typically title)?
    Yes, I used 'dataset' as a resource type and 'Title' to fill the CSW sorting field as recommended.

Q1


<pre>
    the front-end displays only the "search" button (look at capture.png), with nothing else around ; is that a default behaviour? I guess it's not...
</pre>

It is the default behavior: you have to chose the fields you want to use in the catalog you created. In backend, in a catalog (easysdi/catalog/catalogs): you can choose in different default search fields (free text, dates, resources types etc…) and add custom CSW filters (they have to match in lucene indexed fields).

I think what I didn't get is how to configure properly the frond-end module, particularly the search field so as to query the CSW properly...
I join screenshots of my config to target what's wrong in my catalog description.
By, after following your suggestions, the request changed the front-end send is a bit different (I guess it adds the catalog configuration) :
http://192.168.1.82/joomla/index.php/catalogue?view=catalog&search=true&id=1&preview=&jform%5Bsearchtype%5D=simple&jform%5Bid%5D=1#results

Thanks again for the help you've already provided!

Mathieu

Capture1.png (23.1 KB)

Capture2.png (31.7 KB)

geonetwork2015-07-20.log (8.97 KB)

RE: problem front-end catalog display after install (among others) - Added by Blatti Yves over 2 years ago

Hi Mathieu,

Quick but late reply: to edit the search criteria, use the button in the list view here:

keep us informed on the progress

Regards

Capture1_edit.png (24.2 KB)

RE: problem front-end catalog display after install (among others) - Added by Portier Thomas over 2 years ago

Hi Mathieu,

Did you add some resources and metadata in your catalog through the resource panel editor?

Thomas

RE: problem front-end catalog display after install (among others) - Added by Kazmierski Mathieu over 2 years ago

Hi,

OK, so, on Yves' advices, I've succeeded in showing the search form (see ihm_joomla_catalog.png).
Even if I experienced some error in joomla administration while I was trying to publish/unpublish fields (see error.png)

Still the search action ends in no results. :-(

Thomas, what panel are you refering to? The geonetwork panel? In this case, yes, I've harvested some metadata and created/validated/published a "test metadata" as well (see geonetwork_cat.png)...

I add that the geonetwork log highlights some odd lines about "SAXParser Exception", even when I try to send a request through the search button.Perhaps it's just because of empty fields when sending the form, but still...intriguing! :-)

Thanks,
Mathieu

ihm_joomla_catalog.PNG (9.88 KB)

error.png (30.4 KB)

geonetwork_cat.png (46.7 KB)

geonetwork.log (5.04 KB)

RE: problem front-end catalog display after install (among others) - Added by Portier Thomas over 2 years ago

You have to harvest and edit your metadata in EasySDI, not in Geonetwork. EasySDI adds some topics in the metadata in order to manage rights and metadata status. (cf metadata_EasySDI.png)

To edit resources and metadata, you have to create a new menu element in joomla : EasySDI CORE -> Resources. This will let you create resources, metadata and other stuff as diffusion paramters.

To harvest metadata, you have to configure a new CSW virtual service for the service you want to harvest and check "Harvesting". This will be this virtual service that you have to harvest with your geonetwork. The proxy adds some extra content to the metadata during the harvesting.

So it was logical that the result returns nothing.

RE: problem front-end catalog display after install (among others) - Added by Kazmierski Mathieu over 2 years ago

Hi,

OK, first, I've tried to add a Resource menu on my website, but it seems that I can't do anything as super user. The interface throws me a "You are not authorised to view this resource." error. I checked the rights on this module ; I'm supposed to have access to it as a super user, so I don't get the problem. Sorry, it's probably because I'm not really used to Joomla CMS.

About your other point, I'm not really sure of the configuration I set up.
I've created a physical service (see physical_service.png)
Then I've set up a virtual service (see virtual_service.png)...In which, the "harvester" option is checked and can't be unchecked. Is that normal?

Best regards,
mathieu

RE: problem front-end catalog display after install (among others) - Added by Portier Thomas over 2 years ago

Ok, to edit resources, your joomla account must be associated to an EasySDI contact and an organisation. This is in this EasySDI contact that you fix user rules.

So you have to go to Component -> EasySDI -> Organisation -> create your organisation

Then Component -> EasySDI -> Contact -> create your EasySDI contact from your Joomla account and give rules to this user

You'll be able to edit some resources.

For the harvesting, now that you configured your virtual service, you have to harvest it from geonetwork. Did you create a new virtual service or updated the one you use for the catalog?

RE: problem front-end catalog display after install (among others) - Added by Blatti Yves about 2 years ago

Hi Mathieu,

I was having a look at your past issues (missing configurations etc...).

My question is: How did you install easySDI on Joomla ?
  • By using the "upload" feature ?
  • Local folder install ?
  • Web URL install ?
  • By "discovery" having unzipped it before ?
  • Or... any other way ?

The reason is is by using the default upload mechanism we couldn't reproduce your problems.

Thanks for you reply !

RE: problem front-end catalog display after install (among others) - Added by Kazmierski Mathieu about 2 years ago

Hi,

I don't remember accurately, but I reckon I did copy an URL from your website at some point to install it...if it helps.

As I told to Thomas, I'll get back to my test installation in the very next weeks.
Plus, I'll share with you my install process from the start in a doc or smg.

Best regards,
Mathieu

RE: problem front-end catalog display after install (among others) - Added by BRUNET Clément almost 2 years ago

Hi,

I reopen this topic, because, I've exactly the same pb.

My catalog was almost finished and full, I've update my security and softwares patch on my ubuntu server and until the restart of my services, I can not edit or see the metadatas.

In the logs after a long list of my metadata in xml format, I see this exception : net.sf.json.JSONException: JSONObject["infrastructureID"] not found.

I've played the update on the jos_extension table to add the UUID to complete a possible lack...Nothing.

I've search in the geonetwork interface for a "bad metadata not at the easysdi's format", nothing else..

My question is : you, who know well the behavior system, what an simple update could had been broke in my configuration ?

I want to say, which configuration file of tomcat or geonetwork or other could had been crashed and put my catalog down ?

Thank you.
Clément.

RE: problem front-end catalog display after install (among others) - Added by BRUNET Clément almost 2 years ago

Hi, thomas.

Here is the answer.

{'status': 'KO','message': 'Cache invalidation requires an authentication.'}

can you explain it to me ?

RE: problem front-end catalog display after install (among others) - Added by Portier Thomas almost 2 years ago

Did you fill the service, default, invited accounts in the options of the contact component?

RE: problem front-end catalog display after install (among others) - Added by BRUNET Clément almost 2 years ago

Hello,

Thank you Thomas for your answer.

I came back at the office on monday and my catalog was up ! I can't explain why but maybe a sunday server reboot made the config straigth...

I don't really like the mystic dimension of computing..

I stay have a mistake on the display of the logo and the link download.
What I can see in the xsl is that this fields are expected from a rubric named : /sdi:ExtendedMetadata/....

I can't see this rubric in my metadatas, where should it be?
Indeed, I think, a bad geonetwork configuration file stay deployed... Do you have an idea ?

Thank you.
Clément.

(1-15/15)