1M Posts Search - You must see it !!

Discussion in 'Forum Software' started by ChrisTERiS, Aug 2, 2017.

  1. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    How long it takes your forum to search 1 Million posts (all with photos to display in page)?

    If you think that you need $$$ for big brands, special software and plugins, just give a test here:
    http://www.teriakis.com/antibb/index.php

    Use any filters you want by enabling/disabling the buttons on the sidebar.

    And no, I haven't did any magic. Simple jquery/php code and .....smart construction of search engine.

    PS: Just a favor. It's still in testing mode so haven't added lot of security checks in inputs. I'll appreciate a fair use.
     
    Last edited: Aug 2, 2017
  2. TrixieTang

    TrixieTang Politically Incorrect

    8,509
    1,522
    +2,806
    But how would it fare if it were searching actual user-generated content rather than placeholder content? How would it fare if actual users were searching that amount of threads constantly and concurrently?

    It's a nice example of what you're striving to achieve with the search system, but in its current state it says very little about how the final product would actually perform in production use once completed.
     
  3. Paul M

    Paul M Limeade Addict

    3,813
    1,627
    +2,229
    I dont know why a photo on every page would make any difference to a search ?

    As to the question, I dont have 1 million, so a search of 2.5 million posts ;
    (and apparently, I have 161 posts with the word kitten in them :))
     
  4. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    Maybe you haven't seen the full search form? Is not only the button filters (which you can let unused). There are also Search by Username and Search by Text (maybe you mean this by saying user-generated content). Also don't underestimate the value of the "Others" section. Most important of them are "Content with zero replies", "Show only content that I follow", "Show only content from Members that I follow".
     
  5. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    Do you really believe it? Does not makes a difference? Displaying the search results in the page takes the same time when is just text comparing with the time that system needs to load and display a photo? Then why most forum scripts does not display even the user avatar? Search time is not only the time that the engine needs to filter the content but also the time to display the results on te page.

    No problem to prepare a database with 25M posts and not just 2.5M You know why? Because for a text (post) that vBulletin (and xenForo as I tested yesterday) are saving in database with 464 words, my smart code saves it with 150 words only having the EXACT SAME content.
    Give a look here: http://www.teriakis.com/search.php It's just a screen with echo and print_r()
    The first number is the amount of words of a vB saved in search table content. This amount goes down after each "clean" that I'm doing, to endup in 150 words. Now I'm trying to find a way to remove words which exist in both Singular and Plural (eg profile, profiles, user, users etc).

    Unless if you think that content like:
    Code:
    [COLOR=#0000cd][B]
    
    is important to be saved, or duplicate words to be saved etc etc.

    So, yes 0.20 seconds is fast. But with my search engine should takes 0.05 seconds, be sure for it.
     
    Last edited: Aug 3, 2017
  6. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    TrixieTang Paul M

    But both you ignore the most important. That is a "Search as you go" solution. Let me use an example. With the current forum searching style you must set some search criteria, click search and then wait for the results. What happen when you'll see 10.000 results and you want to add more criteria to make the list shorter? You must go back to the form, fill it again, click again the button, wait again the results. Fine now you've 5.000 results only but still need more filters.

    To make the long story short. For each filter that you want to add in current search solution you must go back to search form. Again and again. Here the solution is one click away, on the same screen.
     
  7. markoroots

    markoroots Participant

    70
    13
    +18
    So it's a search plugin? For Vb?
     
  8. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    No, it's a new forum script. That link is just for testing the search engine. The interface should be like this:
    http://www.teriakis.com/cmsforo/

    Please note that is a mockup template and not a working site. But the menu options on the top left should be including in the script. So it's a CMS rather than just a Forum script.
     
  9. markoroots

    markoroots Participant

    70
    13
    +18
    It's wonderful, I like it really more. You are a great!
    I will vote you as Europe president. :)
    Also eWHMCS is wonderful.
     
  10. TrixieTang

    TrixieTang Politically Incorrect

    8,509
    1,522
    +2,806
    Technically, they're two separate things. One is execution time (how long the search actually takes the server), the other is render time (how long the page content takes to display to the user).


    Back when IPB 3 first released it had a feature that let you see a feed of all of a user's recent content in all apps - they had to remove it during the beta testing because the performance wasn't very good once it was tested in an actual production environment.

    You don't know for sure what the bottlenecks will be until you actually have software that can be tested in an actual production environment with actual users generating content and searching.
     
  11. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    This is what I meant. And also the end user does not cares for such details. For what he cares is how fast he/she will see the results on the page with the minimum effort. And back and forh to search form for sure is not what he likes.

    First of all. As I'm clearly stated in my scripts thread, (actually is first on the NOT FOR list), I focus to average webmasters, all those who try with a minimal fee to have as more sections as they can. eg To buy a forum script, with additional: Articles, Classifieds, Reviews, Media Gallery etc etc, you need at least $250. For all those I'm coding this forum. The presentation of the search was just to show that it does not lacks of features.
     
  12. mysiteguy

    mysiteguy Devotee

    2,938
    887
    +2,257
    I welcome additional competition in the forum world, and new ideas for optimization. We'll see how it does once it's released with real data sets, I'll give it a try on a test setup --- assuming I can import the data from one of my big boards.
     
  13. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    The same search engine with more complicated fields works since March 2016 in a Dating site with 20.000 members and at least 150 online users searching. And as you maybe know, search is A-Z for Dating sites. All members are spending their time on that section.

    Sorry no. Is not in my plans to code any importer. Never did in my life even for my vB addons. I know that this out of logic, but sorry this is the way that I used to do.
    But of course I'm not going to put any border to anyone who may wants to code such importer(s).
     

    Attached Files:

  14. Paul M

    Paul M Limeade Addict

    3,813
    1,627
    +2,229
    Yes.

    Yes, but thats displaying the results, which is not relevant to how long the actual search takes.
    Displaying 10 results would take (add) the same time, regardless of how long it took to find the results.

    Thats certainly a nice improvement on paper.
    I hope it works as well for you with real life data and set-ups.

    However, a further note of caution, dont get hung up on such small numbers.
    In practical terms you are not really going to notice the difference between those two figures when performing a search.
    You still have to wait for the results to be sent over the network from the server to you, and then rendered locally by your browser.
     
  15. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    I know it, and I totally agree. As for the others:
    1. I did this post just because I was happy from what I seen. I was expected to be somehow fast, as I did some tricks in search table, but not so much.
    2. The same search engine works in a Dating sites since March 2016 without any problem to support 100+ simultaneus users searching.
    3. But again, as I wrote above, I focus to small sites. As you're also visiting many sites for supporting your mods, most probably you noticed that there are some sites having more categories than threads. I want to give them a choice for something better.
    4. For me mostly counts that I'll be able to release something that I was dreaming years ago, than how many sales I'll have. That's the reason that I'll not release any importer for any forum script (Free or Commercial).
    5. And finally, talking frankly, don't know how long life I've ahead, so no more plans, than being able to release it.
     
  16. rafalp

    rafalp Desu Ex

    1,180
    707
    +363
    Forget real data sets. Wait to see how this will run when people will install this thing on shared providers with their weird configs.
     
    • Pure Genius! Pure Genius! x 1
    • List
  17. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    Already runs in a GoDaddy's (worst provider in my opinion) shared account, and yes, still works that "thing". You know why? Because is soooo smart the search engine my dear.

    It's amazing how people love to drop the wings of anything coming from "anonymous" people. But those who know me, know how much stubborn I'm and how perfectly I can turn the kicks to boomerang.
     
  18. rafalp

    rafalp Desu Ex

    1,180
    707
    +363
    So far this reads like big ego trip on your side with no code or real datasets for us to see how sustainable your claims really are. Hence plenty of us are sceptic to your boasts of being really smart with search engine.

    Thanks but graying lads aint my type ;)
     
  19. ChrisTERiS

    ChrisTERiS Developer

    598
    212
    +349
    rafalp

    After posting my reply before I noticed that you're a Forum developer with your own forum. I wish you the best in your business, but sorry I avoid to start endless convertations with competitors because it's normal that everybody will supports his own "thing".

    And yes, I've a big ego. If you've got your knowledge just by reading books as there were no money for studies, if you were speaking English just by trying to compare the subtitles in movies with the sound, again due to lack of money, be sure that even you (any everybody else) should had ego. But this ego is not the bad one. Better use "Proud".
     
  20. rafalp

    rafalp Desu Ex

    1,180
    707
    +363
    Don't be one of folk who play "you are competitor!" card, especially to guy who does his project 100% open source and without any commercial intent.

    Its very low way to dismiss other.

    Bringing up life experience is ok, but this experience should also teach you that you aren't the only one in the world who self learned language or craft.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.