loads of useful information, examples and tutorials pertaining to web development utilizing asp.net, c#, vb, css, xhtml, javascript, sql, xml, ajax and everything else...

 



Advertise Here
C-Sharpener.com - Programming is Easy!  Learn Asp.Net & C# in just days, Guaranteed!

Universal IQueryable Search Version 2 with Reflection

by naspinski 5/16/2009 2:42:00 AM

'google-like' search with an IQueryable made even simpler

A little bit ago, I came out with this post: Universal IQueryable Search Usable by Linq-to-SQL and Linq-to-Entities and it worked great, but it got me thinking: if I could implement Reflection, it could be even easier to use and require less code to run as it would get the property names/types automatically. So here I am with the new version and it is working great, it has already saved me lots of time. All of the old overloads work as they did previously, I have just added some more. Here is how a few of them work:

single-level search

Now that reflection is being used, you can pass any sorts of objects to the search and it will only compare like-types, no need to specify. If you are trying to search a collection of 'car' objects in an IQueryable named 'cars' you can just do something like this:

Search(object[] keywords)
var results = cars.Search(new object[] {"chevy", 2007});

That will run through all of the properties of 'car', and if any of them are string variables, they will get compared against "chevy" and if they are Int32 variables, they will get compared against 2007. So this search would pull out all 2007 models made by Chevy.

search for specific properties

The above search will search all the properties of an object, but what if 'car' has 100 properties, and you only want to search 2, that is simple as well with this overload:

Search(string[] properties_to_search, object[] keywords)
var results = cars.Search(
  new string[] {"make", "year"},
  new object[] {"chevy", 2007});

Which will only search the properties 'make' and 'year' for the objects; only comparing like Types.

multi-level search

Now let's say that the object 'car' has a some objects that you want to dig into as properties. The 'car' object has an 'engine' property, but if we run the searches above, it will not drill down into the 'engine' object (though it would compare it if you sent an 'engine' as part of the search array). To get it to drill into the engine object, we need to add another parameter, the Type[] types_to_explore so the Search knows to drill down into objects of that type:

Search(object[] keywords, Type[] types_to_explore)
var results = cars.Search(
  new object[] {"V8"}, new Type[] { typeof(engine) });

Now the Search will search anything in the 'car' object and/or in the 'engine' since it now knows to look inside of it; not to mention it will recursively search down into the objects. This would even find something nested like this:
car
  -engine
    -engine
      -style:"V8"

Since you told it to explore all 'engine' types. This would work even if you wanted to search into string, Int32, bool, etc. Types as they all have lots of properties in the world of Reflection - could be a whole new use for this extension?

multi-level search on limited properties

Basically combine what was covered above:

Search(string[] types_to_explore, object[] keywords, Type[] types_to_explore)
var results = cars.Search(
  new string[] {"engine", "style"}, new object[] {"V8"}, new Type[] { typeof(engine) });

I like this one the least as you have to be most verbose (but not as verbose as the other overloads). If you simply try to search for 'style' here, it will not be found, because you did not tell it to search 'engine' (the property, not the type).

so that's the latest

If you haven't looked at the last post, I highly recommend you do, as it is still the most detailed search overloads are explained there (though a bit tougher to use). This is very hard to explain, so I hope you just take a look at the code yourself or better yet, just try it out, you can see with trial and error how it works better than I can explain in a wordy blog. As always, I am very interested in feedback and improvements!
Shout it kick it on DotNetKicks.com

Universal IQueryable Search Usable by Linq-to-SQL and Linq-to-Entities

by naspinski 5/10/2009 10:38:00 AM

ever since Linq came out, I have been improving my simple 'google-like' searches; I think I found the holy grail

UPDATE 05/12/2009 - I have done a ton of updates to the class using Reflection, and a new version which is much more usable and simple will be out in the next day or two.

The 'quick search' is a great tool. Google has shown us that searching with one single, universal search field is the way that people prefer to search. Anyone who has worked with Linq for any amount of time knows this is possible, but it requires a lot of hard-coding and a long jumble of 'where' statements. This class will allow you to run a universal 'google-like' search on any IQueryable.

Now what I am going to show you here was a breakthrough to me, and it has made my life a whole lot easier. It may not be the best way to do this, but it is the best/fastest way I could figure out (hint: I am always looking for improvements), and it is totally portable for *any* multi-column IQueryable collection, plus it does multiple relations as well!

dynamic linq

First of all, this uses the System.Linq.Dynamic dll that was released a while back - it writes a dynamic query each search run, so this was necessary in my approach. If you do not have it, you can get it here: http://msdn2.microsoft.com/en-us/vcsharp/bb894665.aspx.

data for the example

For this example, I am going to be working with Linq-to-SQL, but this works just as well with Linq-to-Entities and have used it in both types of projects. Here is the dummy data I am using for the demo:
CREATE TABLE dbo.category (
category_id  INT NOT NULL PRIMARY KEY IDENTITY,
category_name VARCHAR(MAX)
);

INSERT INTO dbo.category VALUES ('ninja');
INSERT INTO dbo.category VALUES ('pirate');
INSERT INTO dbo.category VALUES ('zombie');
INSERT INTO dbo.category VALUES ('weakling');

CREATE TABLE dbo._user (
_user_id INT NOT NULL PRIMARY KEY IDENTITY,
first_name VARCHAR(MAX),
last_name VARCHAR(MAX),
email_address VARCHAR(MAX),
category_id INT REFERENCES category(category_id)
);

INSERT INTO dbo._user VALUES 
  ('stan', 'naspinski', 'stan@mailinator.com', 2);
INSERT INTO dbo._user VALUES 
  ('sara', 'ortbals', 'hotchick@yahoo.com', 1);
INSERT INTO dbo._user VALUES
  ('rj', 'russell', 'punk@ilovedudes.com', 4);
INSERT INTO dbo._user VALUES 
  ('brian', 'pond', 'captweaksauce@ilovedudes.com', 4);
INSERT INTO dbo._user VALUES 
  ('idaho', 'edokpayi', 'idaho@yahoo.com', 1);
INSERT INTO dbo._user VALUES 
  ('phil', 'robles', 'crazyphil@hotmail.com', 3);
INSERT INTO dbo._user VALUES 
  ('jose', 'meza', 
    'iwishiwasasstrongasstan@gmail.com', 2);

Just a table of users with a referenced table of categories so I can demonstrate the multi-level search. Here is what my dbml looks like:
dbml

quick example

Now, say I want to do a universal search in all the fields for the word 'stan', it is as simple as this:
string[] columns_to_search = {"first_name", "last_name", "email_address", "category.category_name"};
string[] search_terms = {"stan"};
var search_results = new dbDataContext()._users.Search(columns_to_search, search_terms); 

All I did was specify which columns to include in the search, and what terms to search for; doesn't get much simpler than that. This is the simplest approach/overload, so you can get much more detailed if you want, and I will get into that later, but here you can see how powerful and simple this is. Here are the results when bound to a GridView:

search for 'stan'
search for 'stan'

This simply runs through the columns you specify and runs a Contains() against each field; if the keyword is found in one of those fields, it is considered to be a valid record. As you can see, the email address of of record 7 has the word 'stan' in it as well as the name and email address of record 1. Now if I narrow the search by adding another search term: 'pirate' you will see it narrows down the search to just one:

search for {'stan', 'pirate'}
search for 'stan', 'pirate'

Now you can not see 'pirate' in this GridView as it is just an auto-generated GridView, but 'pirate' is the value of the 'category.category_name' record 2 which is referenced in the last column (chopped off a bit); so you can see how easy it is to use relations with this tool.

what is happening

It is pretty simple what is happening. For each object that is passed in the 'keywords' the IQueryable is whittled down so only the ones that 'pass' remain:

searching with 'stan' and 'pirate' (psuedocode)
IQueryable results;
results = dbDataContext._users.Where(any_of_the_specified_fields.Contains(first term));
results = results.Where(any_of_the_specified_fields.Contains(second_term));
results = results.Where(any_of_the_specified_fields.Contains(third_term));
...and so on...

Note that this only does a single db call to get the first IQueryable (in the case your IQueryable is from a database), once that one is in memory, all searches are done against it, so it is pretty fast.

the search textbox

Now that is all very simple, but rarely are your searches going to be hard-coded, so it would be simple to code in a button handler to search through a specified textbox with something like this:
protected void btnSearch_Click(object sender, EventArgs e)
{
  char[] search_delimiters = { ',', ' ', ';', '+' };
  string[] columns_to_search =  {"first_name", "last_name", "email_address", "category.category_name"};
  
  gvSearch.DataSource = new dbDataContext()._users
    .Search( columns_to_search, txtSearch.Text.Split(search_delimiters));
  gvSearch.DataBind();
}

Pretty basic stuff there, you could also get your columns to search from a user interface like a bunch of CheckBoxes or a ListBox control.

getting more specific

If you look into the code, you can see it gets more and more specific the more you drill down into it. With the above overload, it assumes you are using all string comparable fields and all of your inputs are strings. The class's abilities do not stop there, and can get more and more specific. Say you had a massive table with the following fields: int named 'ID', DateTime named 'birthday', text named 'name' and bool named 'male'. If you simply tried the above overload with that, it would error as the types int, DateTime and bool do not have Contains() methods. But, if you specify name/type, you can use this to great advantage:
Dictionary ColumnNamesAndTypes =  new Dictionary<string, Type>();
ColumnNamesAndTypes.Add("ID", typeof(int));
ColumnNamesAndTypes.Add("first_played_videogames", typeof(DateTime));
ColumnNamesAndTypes.Add("name", typeof(string));
ColumnNamesAndTypes.Add("male", typeof(bool));

object[] search_for = {18, 25, DateTime.Parse("12/08/1981"), true, "a","b","c" };

var search_results = largeTable.Search(ColumnNamesAndTypes.ToArray(), search_for);

Now, since the Search now knows the types, it will only run valid types against the fields; for example, the 'true' object will only be run against the 'male' field, as it is the only 'bool' field, the values '18' and '25' are ints, so they will only be compared to the 'age' field of type 'int'. So, what this search would do is return all the people in the IQueryable 'largeTable' that are ages 18 and 25, who 'first_played_videogames' on 12/08/1981, that have an a,b or c in their names who are male. Now I know you can easily write these queries in Linq yourself, but if somehting changes, you are hardcoding a search change that is easy to screw up, this does it all for you.

parsing varying strings

Now there is a little more difficulty with attempting to pass in data through the magical 'google-like' single search box, but it really isn't that hard, check this out:
protected object[] parseSearchString(string search_string)
{
    int obInt;
    DateTime obDt;
    bool obBool;

    char[] search_delimiters = { ',', ' ', ';', '+' };
    List<object> obs = new List<object>();

    foreach (string s in search_string.Split(search_delimiters))
    {
        if (Int32.TryParse(s, out obInt)) obs.Add(obInt);
        else if (DateTime.TryParse(s, out obDt)) obs.Add(obDt);
        else if (bool.TryParse(s, out obBool)) obs.Add(obBool);
        else obs.Add(s); // else it's a string
    }
    return obs.ToArray();
}

Now I obviously didn't go through all the types there, but you get the idea. This takes in a string from a textbox and tries to parse it out to the more specific types before it defaults to a string. Now you might run into conflicts when you want to use integers as strings and such, but if you are this far in this post and still understanding things, I am sure you can figure out a way to code around that. Going back to the original example, now if I wanted to search the '_users' table for either 'category_id' (which is an integer) or the name of that category, I would just use the following:
protected void btnSearch_Click(object sender, EventArgs e)
{
  Dictionary<string, Type> columns_to_search = new Dictionary<string, Type>();
  columns_to_search.Add("category_id", typeof(Int32));
  columns_to_search.Add("category.category_name", typeof(string));
  gvSearch.DataSource = new dbDataContext()._users
    .Search( columns_to_search, parseSearchString(txtSearch.Text));
  gvSearch.DataBind();
}

And as you can see that uses the above 'parseSearchString' so if I enter in '1', it is searched as in Int32:
search for '1'

And if I type in 'ninja' is is searched with a 'string' (ninja is the string value of category_id 1):
search for 'ninja'

that's it

I am hoping that at least a few of you can decipher my ramblings... hopefully you can see how this can be useful to you, and with any luck, it will save you some time and headaches. If people are interested, I can go into the code and how it works, but for now I will just leave you with the source (I even wrote comments... something I am horrible at). I am hopeing someone knows how to get the fields/types automatically from a generic IQueryable so this can be made even better - if you know how to do that, please share!!!

Linq-to-SQL vs Linq-to-Entities : Revisited

by naspinski 5/6/2009 7:24:00 AM

After 2 months of nothing but L2E, I feel I can give a decent comparison

Most likely you have read more than just a few of these comparisons and are tired of the debate. I too disliked L2E from the get-go but decided that my opinion was not totally grounded and that I should really understand L2E before I crap on it, so, I decided to do an entire project at work with it and get down to the nuts and bolts. Now, 2 months, a lot of frustration and some happy customers later, I can give a fully educated opinion on L2E: it is not as bad as I originally thought, in fact, it is pretty sweet in some areas, but still not as easy to use as L2S.

If you have read any of my multiple posts on Entities this will mostly be a re-hash of what I have already said. If not, this is a good summary of my findings and hopefully will help you decide.

dynamic queries

Winner: TIE
Thanks to the geniuses that wrote System.Linq.Dynamic.dll the syntax is *mostly* interchangable between the two languages (which I didn't realize when writing this). If I was forced to pick a winner, it would be L2S as there are more choices for syntax than for L2E, but that can also be viewed as a fault.

linq-to-sql -and- linq-to-entities
var results = db.branch
  .Where("branch_id == @0 || display == @1", 
    new object[] {5, "Hello"});

With L2S, you can do this as well:
var results = db.branch
  .Where("branch_id == 5 || display == \"Hello\"");

working with databound controls

Winner: Linq-to-SQL
This one is a no-brainer. After being lulled into the fact that L2S just works with so many things, I had to work backwards on problems I found with L2E. One annoyance I noticed was that if I was making an update in a GridView, it would error out unless I explicitly had the primary key in the GridView - and if I wanted it hidden, it was necessary to use css; it is documented here.

Another thing I ran across is the fact that you need to use all the required 'Include' statements in order to get a Repeater/GridView to show child values of relationships with Entities where is L2S, you could simply call the relation without any extra work.

multiple databases

Winner: Linq-to-SQL
The choice is quite simple here, as Linq-to-Entities has no native support for multiple databases.

database relation management

Winner: Linq-to-SQL
First of all, simple things such as getting foreign key values with linq-to-entities is much more difficult than in L2S; examples (written as short as I could):

linq-to-sql
int s_id = dataContext.records
  .First(r => r.record_id == 1).state_id;

linq-to-entities
int s_id = Convert.ToInt32(
  Entities.records.First(r => r.record_id == 1)
    .stateReference.EntityKey
    .EntityKeyValues.First().Value
  );

As for putting those relation values into the DB, the clear winner is L2S as well, just take a look at what it takes to set this object 'r' to have a reference to the 'city' which has a key of the integer value of 5:

linq-to-sql
r.city_id = 5;

linq-to-entities
r.cityReference.EntityKey = new EntityKey(
  "YourEntitiesNamespace.cities", "city_id", 5);

a few more...

A while back I went over some other reasons L2E annoyed me such as DataSource annoyances, relations (again) and the pain of updating your model.

verdict

Linq-to-Entities is fully capable and can do *almost* everything Linq-to-SQL can do, it just takes a bit more code some times; the almost is for multiple database utilization. L2E is a great ORM, but many of us were spoiled with L2S, so it is now up to Microsoft to live up to the expectations that they put out for us with this framework that they are deprecating. I hope and assume it will become a superior ORM, but that is in Microsoft's hands now; for the time being, it is not.

Shout it kick it on DotNetKicks.com

Writing Dynamic Linq Queries in Linq-to-Entities

by naspinski 4/29/2009 9:02:00 AM

Believe it or not, L2E actually shines in this area

I am stubbornly plugging along with L2E, and I am even becoming kind of fond of it (in some areas). For example, I really like the way it handles dynamic queries, even more than L2E because you can pass in objects, not just strings! The syntax is a little different, but I think it is easier to understand and a better approach.

setup

First thing is first: you have to use the System.Linq.Dynamic.dll just like with Linq-to_SQL, you can get it from the links on ScottGu's blog. Then just make sure to throw it in your bin folder (or reference it) and include this in the program:
using System.Linq.Dynamic;

how to use 'where'

Where() is a bit different here. In L2S it just takes in a string, but here I will focus on the following overload:
(extension)IQueryable<T> IQueryable<T>
  .Where(string predicate, params object[] values)

Notice it does take in a string, but also the object array. So say I want to recreate this query:
var results = db.branch
  .Where(b => b.branch_id == 5 || b.display == "Hello");

With a dynamic query I would do this:
var results = db.branch
  .Where("branch_id == @0 || display == @1", 
    new object[] {5, "Hello"});

See how easy that was? The @0 and @1 are simply replaced with the corresponding objects in the array. Also notice that the array takes in all sorts of different object types as it does not matter what you are putting through. Now obviously you aren't going to set the array manually every time, that would defeat the purpose of the dynamic query, but you can easily see how this works.

In some programs, I have even built the string part (myQuery in this example) of the query dynamically with a running count so I know what @ to place in the string; along with the string being built I would add the objects to a List<object> myObjects and simply call the final products with:
var results = db.branch
  .Where(myQuery, myObjects.ToArray());

how to use 'select'

This is identical to L2S, very simple. If you don't use a Select(), it will just grab all the full objects. But if you want to grab select parts, or change names of some stuff, it is not difficult. A simple dynamic Select could look like this:
var results = db.branch
  .Select("new(branch_id,display)");

That would just grab the branch_id and display values of the branch items. But you can also change the name of the column they are represented in which can often be useful in dynamically produced tables and such:
var results = db.branch
  .Select("new(branch_id as BRANCH,display as NINJAS)");

Those columns will now be titled 'BRANCH' and 'NINJAS' respectively. Not too tough there. I think L2E got this dynamic query thing pretty spot on.

Tags: ,

c# | entities

Store update, insert, or delete statement affected an unexpected number of rows (0). Entities may have been modified or deleted since entities were loaded. Refresh ObjectStateManager entries.

by naspinski 4/21/2009 12:24:00 PM

Confusing error while using an EntityDataSource with a GridView update

Linq-to-Entities baffled me yet again today. I was simply trying to bind an EntityDataSource to a GridView then do an update. It worked as expected, even with the foreign keys... but that was only if I used the auto-generated columns. When I tried to make the fields myself with TemplateFields, all hell broke loose and this error popped up.

While this may make sense to some people, it made none to me, so I basically just started changing things one-by-one until something broke. When I changed AutoGenerateColumns to 'false' it would break. But... if I added in a BoundField that had the primary key in it, it would work? Ok, great, but I didn't want the user to see the primary key field, they don't care about it, so I set that BoundField to Visible="false"; it broke again.

solution

You need to explicitly include a BoundField of the primary key! Since I didn't want the user to see my primary key, I had to resort to hiding it via css:

<asp:BoundField DataField="my_primary_key" ItemStyle-CssClass="hidden" HeaderStyle-CssClass="hidden" />


Where 'hidden' is a class in css that has it's display set to 'none'. Not sure why Entities can't figure this out like all the other DataSources...