Close Menu
Ztoog
    What's Hot
    Mobile

    In North America, the Pixel’s Q4 market share tripled in 2023 from 2021

    AI

    Taipy vs Streamlit: Navigating the Best Path to Build Python Data & AI Web Applications with Multi-user Capability, Large Data Support, and UI Design Flexibility

    Crypto

    Shiba Inu Nears Make-Or-Break Resistance Barrier

    Important Pages:
    • About Us
    • Contact us
    • Privacy Policy
    • Terms & Conditions
    Facebook X (Twitter) Instagram Pinterest
    Facebook X (Twitter) Instagram Pinterest
    Ztoog
    • Home
    • The Future

      How I Turn Unstructured PDFs into Revenue-Ready Spreadsheets

      Is it the best tool for 2025?

      The clocks that helped define time from London’s Royal Observatory

      Summer Movies Are Here, and So Are the New Popcorn Buckets

      India-Pak conflict: Pak appoints ISI chief, appointment comes in backdrop of the Pahalgam attack

    • Technology

      Ensure Hard Work Is Recognized With These 3 Steps

      Cicada map 2025: Where will Brood XIV cicadas emerge this spring?

      Is Duolingo the face of an AI jobs crisis?

      The US DOD transfers its AI-based Open Price Exploration for National Security program to nonprofit Critical Minerals Forum to boost Western supply deals (Ernest Scheyder/Reuters)

      The more Google kills Fitbit, the more I want a Fitbit Sense 3

    • Gadgets

      Maono Caster G1 Neo & PD200X Review: Budget Streaming Gear for Aspiring Creators

      Apple plans to split iPhone 18 launch into two phases in 2026

      Upgrade your desk to Starfleet status with this $95 USB-C hub

      37 Best Graduation Gift Ideas (2025): For College Grads

      Backblaze responds to claims of “sham accounting,” customer backups at risk

    • Mobile

      Samsung Galaxy S25 Edge promo materials leak

      What are people doing with those free T-Mobile lines? Way more than you’d expect

      Samsung doesn’t want budget Galaxy phones to use exclusive AI features

      COROS’s charging adapter is a neat solution to the smartwatch charging cable problem

      Fortnite said to return to the US iOS App Store next week following court verdict

    • Science

      Failed Soviet probe will soon crash to Earth – and we don’t know where

      Trump administration cuts off all future federal funding to Harvard

      Does kissing spread gluten? New research offers a clue.

      Why Balcony Solar Panels Haven’t Taken Off in the US

      ‘Dark photon’ theory of light aims to tear up a century of physics

    • AI

      How to build a better AI benchmark

      Q&A: A roadmap for revolutionizing health care through data-driven innovation | Ztoog

      This data set helps researchers spot harmful stereotypes in LLMs

      Making AI models more trustworthy for high-stakes settings | Ztoog

      The AI Hype Index: AI agent cyberattacks, racing robots, and musical models

    • Crypto

      ‘The Big Short’ Coming For Bitcoin? Why BTC Will Clear $110,000

      Bitcoin Holds Above $95K Despite Weak Blockchain Activity — Analytics Firm Explains Why

      eToro eyes US IPO launch as early as next week amid easing concerns over Trump’s tariffs

      Cardano ‘Looks Dope,’ Analyst Predicts Big Move Soon

      Speak at Ztoog Disrupt 2025: Applications now open

    Ztoog
    Home » New method accelerates data retrieval in huge databases | Ztoog
    AI

    New method accelerates data retrieval in huge databases | Ztoog

    Facebook Twitter Pinterest WhatsApp
    New method accelerates data retrieval in huge databases | Ztoog
    Share
    Facebook Twitter LinkedIn Pinterest WhatsApp

    Hashing is a core operation in most on-line databases, like a library catalogue or an e-commerce web site. A hash operate generates codes that straight decide the situation the place data can be saved. So, utilizing these codes, it’s simpler to search out and retrieve the data.

    However, as a result of conventional hash capabilities generate codes randomly, generally two items of data could be hashed with the identical worth. This causes collisions — when looking for one merchandise factors a person to many items of data with the identical hash worth. It takes for much longer to search out the best one, ensuing in slower searches and diminished efficiency.

    Certain forms of hash capabilities, often called excellent hash capabilities, are designed to position the data in a approach that stops collisions. But they’re time-consuming to assemble for every dataset and take extra time to compute than conventional hash capabilities.

    Since hashing is used in so many purposes, from database indexing to data compression to cryptography, quick and environment friendly hash capabilities are vital. So, researchers from MIT and elsewhere got down to see if they might use machine studying to construct higher hash capabilities.

    They discovered that, in sure conditions, utilizing discovered fashions as an alternative of conventional hash capabilities might end result in half as many collisions. These discovered fashions are created by operating a machine-learning algorithm on a dataset to seize particular traits. The workforce’s experiments additionally confirmed that discovered fashions have been usually extra computationally environment friendly than excellent hash capabilities.

    “What we found in this work is that in some situations we can come up with a better tradeoff between the computation of the hash function and the collisions we will face. In these situations, the computation time for the hash function can be increased a bit, but at the same time its collisions can be reduced very significantly,” says Ibrahim Sabek, a postdoc in the MIT Data Systems Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL).

    Their analysis, which will likely be introduced on the 2023 International Conference on Very Large Databases, demonstrates how a hash operate could be designed to considerably velocity up searches in a huge database. For occasion, their approach might speed up computational programs that scientists use to retailer and analyze DNA, amino acid sequences, or different organic data.

    Sabek is the co-lead creator of the paper with Department of Electrical Engineering and Computer Science (EECS) graduate pupil Kapil Vaidya. They are joined by co-authors Dominik Horn, a graduate pupil on the Technical University of Munich; Andreas Kipf, an MIT postdoc; Michael Mitzenmacher, professor of pc science on the Harvard John A. Paulson School of Engineering and Applied Sciences; and senior creator Tim Kraska, affiliate professor of EECS at MIT and co-director of the Data, Systems, and AI Lab.

    Hashing it out

    Given a data enter, or key, a conventional hash operate generates a random quantity, or code, that corresponds to the slot the place that key will likely be saved. To use a easy instance, if there are 10 keys to be put into 10 slots, the operate would generate an integer between 1 and 10 for every enter. It is extremely possible that two keys will find yourself in the identical slot, inflicting collisions.

    Perfect hash capabilities present a collision-free various. Researchers give the operate some additional information, such because the variety of slots the data are to be positioned into. Then it might carry out further computations to determine the place to place every key to keep away from collisions. However, these added computations make the operate more durable to create and fewer environment friendly.

    “We were wondering, if we know more about the data — that it will come from a particular distribution — can we use learned models to build a hash function that can actually reduce collisions?” Vaidya says.

    A data distribution reveals all doable values in a dataset, and the way usually every worth happens. The distribution can be utilized to calculate the likelihood {that a} explicit worth is in a data pattern.

    The researchers took a small pattern from a dataset and used machine studying to approximate the form of the data’s distribution, or how the data are unfold out. The discovered mannequin then makes use of the approximation to foretell the situation of a key in the dataset.

    They discovered that discovered fashions have been simpler to construct and sooner to run than excellent hash capabilities and that they led to fewer collisions than conventional hash capabilities if data are distributed in a predictable approach. But if the data will not be predictably distributed as a result of gaps between data factors differ too broadly, utilizing discovered fashions may trigger extra collisions.

    “We may have a huge number of data inputs, and the gaps between consecutive inputs are very different, so learning a model to capture the data distribution of these inputs is quite difficult,” Sabek explains.

    Fewer collisions, sooner outcomes

    When data have been predictably distributed, discovered fashions might cut back the ratio of colliding keys in a dataset from 30 % to fifteen %, in contrast with conventional hash capabilities. They have been additionally capable of obtain higher throughput than excellent hash capabilities. In the very best instances, discovered fashions diminished the runtime by practically 30 %.

    As they explored the usage of discovered fashions for hashing, the researchers additionally discovered that throughput was impacted most by the variety of sub-models. Each discovered mannequin consists of smaller linear fashions that approximate the data distribution for various components of the data. With extra sub-models, the discovered mannequin produces a extra correct approximation, however it takes extra time.

    “At a certain threshold of sub-models, you get enough information to build the approximation that you need for the hash function. But after that, it won’t lead to more improvement in collision reduction,” Sabek says.

    Building off this evaluation, the researchers wish to use discovered fashions to design hash capabilities for different forms of data. They additionally plan to discover discovered hashing for databases in which data could be inserted or deleted. When data are up to date in this fashion, the mannequin wants to vary accordingly, however altering the mannequin whereas sustaining accuracy is a troublesome drawback.

    “We want to encourage the community to use machine learning inside more fundamental data structures and algorithms. Any kind of core data structure presents us with an opportunity to use machine learning to capture data properties and get better performance. There is still a lot we can explore,” Sabek says.

    “Hashing and indexing functions are core to a lot of database functionality. Given the variety of users and use cases, there is no one size fits all hashing, and learned models help adapt the database to a specific user. This paper is a great balanced analysis of the feasibility of these new techniques and does a good job of talking rigorously about the pros and cons, and helps us build our understanding of when such methods can be expected to work well,” says Murali Narayanaswamy, a principal machine studying scientist at Amazon, who was not concerned with this work. “Exploring these kinds of enhancements is an exciting area of research both in academia and industry, and the kind of rigor shown in this work is critical for these methods to have large impact.”

    This work was supported, in half, by Google, Intel, Microsoft, the U.S. National Science Foundation, the U.S. Air Force Research Laboratory, and the U.S. Air Force Artificial Intelligence Accelerator.

    Share. Facebook Twitter Pinterest LinkedIn WhatsApp

    Related Posts

    AI

    How to build a better AI benchmark

    AI

    Q&A: A roadmap for revolutionizing health care through data-driven innovation | Ztoog

    AI

    This data set helps researchers spot harmful stereotypes in LLMs

    AI

    Making AI models more trustworthy for high-stakes settings | Ztoog

    AI

    The AI Hype Index: AI agent cyberattacks, racing robots, and musical models

    AI

    Novel method detects microbial contamination in cell cultures | Ztoog

    AI

    Seeing AI as a collaborator, not a creator

    AI

    “Periodic table of machine learning” could fuel AI discovery | Ztoog

    Leave A Reply Cancel Reply

    Follow Us
    • Facebook
    • Twitter
    • Pinterest
    • Instagram
    Top Posts
    AI

    Meet ChemBench: A Machine Learning Framework Designed to Rigorously Evaluate the Chemical Knowledge and Reasoning Abilities of LLMs

    The surge in synthetic intelligence analysis has heralded a brand new period throughout numerous scientific…

    Science

    A comet wagged its tail as it flew past the sun

    A gust of photo voltaic wind appeared to make a comet wiggle its tail as…

    Science

    Which animals can and can’t fart? 

    Who amongst us hasn’t questioned if a snake can fart?  Dani Rabiotti, a quantitative ecologist,…

    Crypto

    When Is the Next BTC Halving Date? Bitcoin Halving Guide

    Guide: What Is The Bitcoin (BTC) Halving? The anticipation round the subsequent BTC halving is…

    Gadgets

    This Nespresso machine is a perfect last-minute gift. Get it 34% off at Amazon

    We could earn income from the merchandise out there on this web page and take…

    Our Picks
    Technology

    Socar’s ride-hailing platform Tada adds to growing list of tech layoffs 

    Mobile

    Weekly poll: do you use Android Auto or Apple CarPlay?

    Mobile

    Verizon, Sideline, Google Voice, and more

    Categories
    • AI (1,482)
    • Crypto (1,744)
    • Gadgets (1,796)
    • Mobile (1,839)
    • Science (1,853)
    • Technology (1,789)
    • The Future (1,635)
    Most Popular
    Gadgets

    15 Best Mattresses You Can Buy Online (2023)

    Crypto

    Bitcoin Transaction Volumes Hit New Highs Amid Signs Of A Supply Shock

    Science

    Why don’t we remember being a baby? New clues in memory mystery.

    Ztoog
    Facebook X (Twitter) Instagram Pinterest
    • Home
    • About Us
    • Contact us
    • Privacy Policy
    • Terms & Conditions
    © 2025 Ztoog.

    Type above and press Enter to search. Press Esc to cancel.