Semantic technology/architecture for image collections (for a web app)

manpreet Best Answer 2 years ago

Being new to semantic techs and having read much about semantic representation structures (concept nets, (xml) topic maps, semantic nets, ontologies, description logic, as well as Semantic Web techslike RDF(S), OWL(S), SPARQL), I'm currently quite confused what semantic tech to use for my current app requirement:

Goal: a web app using crowd-sourcing to semantically describe the contents of photos in photo collections -- i.e. to describe the depicted "scenes" in terms of observable subjects and objects, their respective appearances (=characteristic features), their associations, their actions, etc. -- in a textual (tag-based) manner so that a feature-based semantic search (initial) and navigation (afterwards) paradigm is supported.

Undoubtedly, I'll need to enhance recent folksonomic (tagging) approaches by a semantic tech to organize and browse the contents -- a tech nearly as simple and flexible/dynamic as social tagging and powerful enough to formalize the required statements. I think most of these constructs should be supported:

1. Concepts, concept types, and concept instances.
   Respective system-supported relations such as is-a, is-instance-of, is-subtype-of, etc.

   Examples:

   • 'Man', 'Woman' is-a 'Person', 'Person' is-a 'Animate', etc.
   • 'Peter Parker' is-instance-of 'Man'. 'Mary Jane' is_instance_of 'Woman'.
   • 'Brasilia' is-city-of 'Brazil' located-in 'South America'. 
     
     

2. Concept features/properties (= typed attributes and relations) with system- and user-defined names (the diff. kinds).

3. [concerning the attributes]:

   • Attributes of simple data type, as well as
   • Attributes of complex type (composites; ref. to concept or concept type), i.e. 
     system supports 'has-part' relation.
   • Perhaps the distinction between 'single-' and 'multi-valued' attributes.
     
     

   Examples:

   • 'Person' has attribute 'last-name' of type 'string', 'age' of type 'int',etc. for:
     Concept 'Person Parker' first-name 'Peter', age '29'.
   • 'Person' has relation 'knows' to another 'Person' , e.g. for: 
     'Peter Parker' knows 'Mary Jane Watson'.
   • 'Peter Parker' wears 'body suit' of-color 'red and blue'. 
     
     

4. [concerning the relations]:

   • Mostly binary BUT also some cases of n-ary relations, e.g.
     ternary rel "cuts_with(Person,Object,Tool)" for expressing "Peter cuts bread with knife".
     So, actually we have hyper-graphs, but higher-order rels could be handled through multiple binary rels (reification).

   • domain and/or range restrictions for relations: e.g.
     Relation 'has-human-part' goes from concept 'Person' to concept 'HumanPart'.

   • relations on relations, in other words: secondary statements on primary statements. E.g.:
     "'Harry Osborn' suspects ('Peter Parker' knows 'Spiderman')" -- i.e. a combined/higher-order range. The other case: "('Plastic' x 'Metal') is-glued-by 'mySuperGlue' (instance of Glue)" -- i.e. a combined/higher-order domain.

5. Topology/location-based descriptions, e.g.

   • 'Mary Jane' stands-behind 'Peter'.
   • The river 'abc' is-to-the-south-of church 'xyz'.
   • 'Shark swarm' is-on-the-upper-right (of the image). 
     
     

So my main question basically is:
Q1: What semantic representation technology would you use for this web app context?

• Would you go with a special semantic network type?
  Which (optimally light-weight) type would be powerful enough?
  
  Or, would you instead go with a Semantic Web tech like RDF(S) or OWL(S)?
  In this case, which one would at least be needed?
  
  
• Which storage kind would this selected semantic tech use or be appropriate?
  RDBMS, Graph DBs, or Triple Stores?

Q2: Do you know any good similar project(s) you could me point to?

Thank you all very much for you suggestions.