Package 

Object RobotRulesParser.Companion

    • Method Summary

      Modifier and Type Method Description
      final Unit main(Array<String> argv) command-line main for testing
      final Logger getLOG()
      final BaseRobotRules getEMPTY_RULES() A BaseRobotRules object appropriate for use when the robots.txt file is empty or missing; all requests are allowed.
      final Hashtable<String, BaseRobotRules> getCACHE()
      final BaseRobotRules getFORBID_ALL_RULES() A BaseRobotRules object appropriate for use when the robots.txt file is not fetched due to a 403/Forbidden response; all requests are disallowed.
      final Unit setFORBID_ALL_RULES(BaseRobotRules FORBID_ALL_RULES) A BaseRobotRules object appropriate for use when the robots.txt file is not fetched due to a 403/Forbidden response; all requests are disallowed.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • getLOG

         final Logger getLOG()

      • getEMPTY_RULES

         final BaseRobotRules getEMPTY_RULES()

        A BaseRobotRules object appropriate for use when the robots.txt file is empty or missing; all requests are allowed.

      • getFORBID_ALL_RULES

         final BaseRobotRules getFORBID_ALL_RULES()

        A BaseRobotRules object appropriate for use when the robots.txt file is not fetched due to a 403/Forbidden response; all requests are disallowed.

      • setFORBID_ALL_RULES

         final Unit setFORBID_ALL_RULES(BaseRobotRules FORBID_ALL_RULES)

        A BaseRobotRules object appropriate for use when the robots.txt file is not fetched due to a 403/Forbidden response; all requests are disallowed.