Best Approach for Custom Information Extraction (NER)

manpreet Best Answer 2 years ago

I'm trying to extract locations from blobs of text (NER/IE) and have tried many solutions all which are far too innaccurate spacy, Stanford etc etc.

All really are only about 80-90% accurate on my dataset (spacy was like 70%), another problem I'm having is not having a probability that means anything for these entities so I don't know confidence and can't proceed accordingly.

I tried a super naive approach of splitting my blobs into singular words then extracting surrounding context as features, also used a location placename lookup (30/40k location placenames) as a feature aswell. Then I used just a classifier(XGDBoost) and the results where much better once I trained the classifier on about 3k manually labelled datapoints (100k total only 3k where locations). 95% precision for states/countries and about 85% for cities.

This approach sucks obviously but why is it outperforming everything I have tried? I think the black box approach to NER just isn't working for my data problem, I tried spacy custom training and it really just didn't seem like it was going to work. Not having a confidence in the entity is kind of killer also as the probability they give you for that is almost meaningless.

Is there someway I can approach this problem a little better to improve my results even more? shallow nlp for like 2/3/4-grams? Another problem I have with my approach is the output of the classifier isnt some sequential entity, its literally just classified word blobs which somehow need to be clustered back into one entity i.e : -> San Francisco, CA is just 'city','city', '0','state' with no concept of them being the same entity

spacy example:

example blob:

About Us - Employment Opportunities Donate Donate Now The Power of Mushrooms Enhancing Response Where We Work Map Australia Africa Asia Pacific Our Work Agriculture Anti - Trafficking and Gender - based Violence Education Emergency Response Health and Nutrition Rural and Economic Development About Us Who We Are Annual Report Newsletters Employment Opportunities Video Library Contact Us Login My Profile Donate Join Our Email List Employment Opportunities Annual Report Newsletters Policies Video Library Contact Us Employment Opportunities Current Career Opportunity Internships Volunteer Who We Are Our History Employment Opportunities with World Hope International Working in Service to the Poor Are you a professional that wants a sense of satisfaction out of your job that goes beyond words of affirmation or a pat on the back ? You could be a part of a global community serving the poor in the name of Jesus Christ . You could use your talents and resources to make a significant difference to millions . Help World Hope International give a hand up rather than a hand out . Career opportunities . Internship opportunities . Volunteer Why We Work Here World Hope International envisions a world free of poverty . Where young girls aren ’ t sold into sexual slavery . Where every child has enough to eat . Where men and women can earn a fair and honest wage , and their children aren ’ t kept from an education . Where every community in Africa has clean water . As an employee of World Hope International , these are the people you will work for . Regardless of their religious beliefs , gender , race or ethnic background , you will help shine the light of hope into the darkness of poverty , injustice and oppression . Find out more by learning about the of World Hope International and reviewing a summary of our work in the most recent history annual report . Equal Opportunity Employer World Hope International is both an equal opportunity employer and a faith - based religious organization . We hire US employees without regard to race ,
                                                

                                                                                                        REPLY
                                                    
                                                
                                                


                                                    

                                                        
                                                            0
                                                            views 

                                                            0
                                                                likes 
                                                            
                                                            

                                                            0
                                                            shares

                                                            


    

         Facebook
        
         Twitter
        
        
         Linked In 
        
          WhatsApp